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1.  Introduction 


1.1  Background 

1.1.1  The  Land  Mine  Threat 

Since  World  War  II,  land  mines  have  posed  a  major  threat  to  U.S.  ground  forces.  An  effective 
force  multiplier,  land  mines  play  an  especially  important  role  in  the  asymmetric  warfare  in  which 
the  U.S.  has  engaged  in  conflicts  of  the  past  15  years.  Hambric  and  Schneck  (1996)  provided  an 
assessment  of  the  land  mine  threat  stating,  “Mines  are  a  major  threat  in  all  types  of  combat  and 
will  be  the  major  threat  in  operations  other  than  war. .  .which  are  expected  to  be  the  most  likely 
missions  for  U.S.  forces  in  the  future. .  .The  widespread  employment  of  land  mines  threatens  to 
neutralize  U.S.  advantages  in  firepower  and  mobility  by  severely  limiting  our  ability  to  maneuver 
and  by  disrupting  our  tactical  synchronization.”  Subsequent  assessments  from  Operations  Joint 
Endeavor  (U.S.  GAO,  2001;  Schneck  &  Green,  1997)  and  Enduring  Freedom  (LaMoe  &  Read, 
2002)  have  validated  this  appraisal  and  suggest  that  its  severity  may  have  been  underestimated. 
Moreover,  the  counter-mine  problem  seen  in  the  contemporary  operational  environment  was 
predicted  to  worsen  (LaMoe  &  Read,  2002).  Kern’s  (2005)  recent  assessment  of  the  threat  echoes 
that  of  LaMoe  and  Read  (2002): 

We  must  take  this  challenge  [to  defeat  mine  threats]  seriously  and  field  the  best 
solutions  quickly  to  protect  our  forces.  We  must  learn  how  to  detect  at  distance 
and  speed  to  ensure  our  future  operational  concepts  are  not  derailed  by  the  old 
simple  devices. .  .this  problem  demands  our  best  work  now. .  .asymmetric  warfare 
using  mines  of  multiple  varieties. 

1.1.2  Progress  in  Land  Mine  Detection 

Land  mine  detection  is  the  core  task  in  counter-mine  operations.  Discovering  the  locations  of 
hidden  mines  remains  an  extremely  hazardous  task  because  it  puts  dismounted  troops  who  are 
operating  hand-held  detection  equipment  close  to  buried  and  thus  hidden,  live,  lethal  ordnance. 

Significant  advances  have  been  achieved  recently  in  detection  capability,  related  to  the  develop¬ 
ment  and  fielding  of  new  equipment  and  training.  Current  systems  are  less  than  perfect,  however, 
and  further  work  on  detection  is  needed  to  protect  our  troops  and  ensure  maneuver  capability. 

The  recent  deployment  of  a  new  hand-held  detection  system,  the  Army-Navy/Portable  Special 
Search  (AN/PSS)-14  (see  figure  1)  represents  a  major  technological  advance.  The  first  detector 
to  combine  two  sensors,  an  electromagnetic  induction  subsystem,  and  ground-penetrating  radar 
(GPR),  this  system  has  been  deployed  for  use  in  Operations  Enduring  Freedom  and  Iraqi 
Freedom  and  has  received  favorable  feedback  from  its  users  in  the  field  (82nd  Airborne,  2003). 
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Figure  1.  AN/PSS-14  operated  near  Bagram  Airport,  Afghanistan,  in  2004. 


Significant  progress  has  also  been  made  in  training  troops  to  operate  the  AN/PSS-14  and  its 
predecessor,  the  AN/PSS-12,  both  of  which  are  employed  in  current  operations.  Training 
programs  based  on  cognitive  science  research  have  raised  detection  rates  sufficiently  beyond 
those  produced  by  previous  training  programs  (Staszewski  &  Davison,  2002),  resulting  in  the 
Army’s  adoption  of  these  programs  to  train  AN/PSS-12  and  AN/PSS-14  operators.  As  demon¬ 
strated  in  the  initial  stages  of  system  development  and  recently  confirmed  in  the  field,  the 
performance  of  the  AN/PSS-14  and  dismounted  mine  detection  capability  depends  crucially 
on  the  skills  of  its  operator. 

1.1.3  Transfer  of  Training  to  the  Operational  Environment 

An  in-theater  assessment  of  AN/PSS-14  following  its  deployment  in  support  of  Operation  Enduring 
Freedom  addressed  issues  about  the  AN/PSS-14’s  capabilities  and  use  in  the  operational  environ¬ 
ment  (Maurer,  2003).  Evaluators  concluded  that  the  equipment  was  performing  to  expectations: 

(a)  Soldiers  could  safely  detect  very  low  metal  mines  from  flush  emplacement  to  depths  of  4  inches; 

(b)  Soldiers  could  detect  low  metal  mines  close  to  metal  clutter  and  in  metal-laden  soils;  (c)  Soldiers 
reported  confidence  that  they  could  detect  low  metal  mines  using  the  AN/PSS-14. 

Evaluators’  conclusions  contained  an  important  caveat  related  to  operator  skill,  however.  In¬ 
theater  testing  showed  that  the  techniques  of  most  operators  were  deficient  after  a  month  or  more 
without  system  use,  resulting  in  problematic  detection  rates  (73.6%).  Fortunately,  delivery  of 
refresher  training  restored  performance  to  desirable  levels  (98.6%)  and  increased  operators’ 
confidence  in  their  detection  capabilities.  These  observations  underscored  the  critical  role  of 
operator  training  and  skill  and  implied  that  many  operators’  skills  had  deteriorated  in  the  interval 


2 


between  initial  training  and  operational  use  of  the  AN/PSS-14.  Moreover,  evaluators  observed 
that  noncommissioned  officer  (NCO)  personnel  with  supervisory  responsibilities  who  were 
unfamiliar  with  operator  training  and  proper  operational  technique  lacked  the  knowledge 
required  to  detect  these  deficiencies. 

Another  later  in- theater  assessment  of  operator  skills  conducted  by  U.S.  Army  Engineer  School 
(USAES)  personnel  also  showed  that  Soldiers’  detection  rates  had  fallen  substantially  below  the 
standard  for  successful  completion  of  AN/PSS-14  training  (Mincey,  2005).  Evaluation  of 
operators’  techniques  revealed  major  deficiencies.  This  decrement  was  also  attributed  to  an 
unspecified  interval  without  practice  between  Soldiers’  completion  of  initial  new  equipment 
training  (NET)  and  subsequent  in-theater  evaluation. 

1.1.4  Skill  Decay 

The  phrase  “use  it  or  lose  it”  accurately  describes  the  fate  of  skills  that  go  unpracticed.  Extensive 
scientific  literature  supports  this  saying  (Healy  &  Bourne,  1995;  Farr,  1987;  Rose,  Czamolewski, 
Gragg,  Austin,  Ford,  Doyle,  &  Hagman,  1985;  Shields,  Goldberg,  &  Dressel,  1979).  Generally, 
the  longer  the  interval  without  practice,  the  greater  the  loss  in  proficiency.  Although  the  length  of 
the  retention  interval  plays  a  critical  role  in  predicting  how  much  proficiency  is  likely  to  suffer,  the 
level  of  skill  at  the  start  of  the  retention  interval  is  also  an  important  determinant  of  the  deteriora¬ 
tion  that  occurs  with  the  absence  of  task  practice  (Anderson  &  Schunn,  1995;  Bahrick  &  Hall, 
1991).  It  follows  that  efforts  to  understand  and  mitigate  skill  decay  must  examine  the  original 
level  of  learning  achieved  and  the  effects  of  denial  of  practice. 

This  reasoning  applied  to  the  in-theater  performance  of  AN-PSS-14  operators  motivated  (1)  the 
assessment  of  operator  training  reported  here  and  (2)  a  subsequent  experimental  investigation  of 
skill  retention  in  progress. 

1.2  Problem  Statement 

Evaluations  of  Soldiers  trained  to  use  the  U.S.  Army’s  AN/PSS-14  mine  detector  in  support  of 
ongoing  operations  suggested  that  operator  skills  decayed  between  completion  of  original 
training  and  operational  use.  Such  losses  can  compromise  our  forces’  counter-mine  capability, 
threaten  mission  success,  and  jeopardize  operators  and  other  mission-related  personnel.  An 
understanding  of  the  factors  contributing  to  this  problem  is  required  for  the  formulation  of 
mitigating  policies.  These  factors  include  the  level  of  skill  that  novice  operators  acquire  in 
AN/PSS-14  NET. 

The  U.S.  Army  Engineer  School,  Training  and  Doctrine  Command  (TRADOC)  System  Manager 
for  Assured  Mobility  (TSM-AM),  Counter-Explosives  Hazards  Center  (CEHC)  provided  the 
AN/PSS-14  NET.  The  training  program  consisted  of  40  hours’  classroom  instruction  and  hands- 
on  training.  Typically,  three  NCOs  were  in  charge  of  the  training,  along  with  training  assistance 
provided  by  Cyterra  or  Battelle  contractors  (or  both).  The  training  duration  evolved  from  the 
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initial  40-hour  block  of  training  employed  for  testing  the  AN/PSS-14  prototype,  the  Hand-Held 
Stand-Off  Mine  Detection  System  (HSTAMIDS),  in  2000.  Its  effectiveness  was  confirmed  by 
its  inclusion  of  successful  operational  tests  of  the  system  (Santiago,  Locke,  &  Reidy,  2004). 
Modifications  of  the  system  have  followed,  including  incorporation  of  an  algorithm  that  supports 
the  aided  target  recognition  function,  but  the  40-hour  duration  of  training  has  not  been  changed. 
Effects  of  varying  the  training  time  or  examining  its  adequacy  or  efficiency  of  AN/PSS-14 
training  have  not  been  investigated. 

1.3  Objectives 

The  TSM-AM  and  the  USAES  Director  of  Training  commissioned  this  effort  to  evaluate  current 
AN/PSS-14  NET  delivered  by  USAES,  TSM-AM,  CEHC  and  to  identify  and  recommend  means 
to  maximize  its  effectiveness  and  efficiency.  A  specific  issue  for  USAES  leaders  was  whether 
the  40-hour-long  training  program  was  needed  to  train  Soldiers  to  meet  the  current  standards.  A 
related  task  was  to  determine  the  amount  of  training  that  Soldiers  need  to  achieve  current  per¬ 
formance  standards  for  USAES  operator  certification;  current  records  indicated  that  these 
standards  were  being  met  with  the  40-hour  program  of  instruction. 

The  scope  of  inquiry  was  comprehensive.  It  included  consideration  of  the  training  infrastructure, 
including  the  site,  facilities,  training  and  testing  aids,  personnel  needs,  and  procedures.  Issues 
for  examination  included 

a.  What  were  the  infrastructure  requirements  needed  for  training  Soldiers  to  use  this  mine 
detector?  How  did  the  design  of  the  training  site  and  available  resources  impact  training? 

b.  How  efficient  was  the  training?  Were  significant  amounts  of  time  being  lost  in  teaching 
topics  that  were  not  relevant?  Could  more  efficient  ways  be  established  to  teach  the 
different  tasks  and  subtasks  in  mine  detection? 

c.  What  areas  were  being  over-  or  under-taught,  and  how  should  that  guide  reallocation  of 
training  time  to  drills  on  the  basis  of  need? 

1.4  Approach 

The  evaluation  approach  was  based  on  an  empirical  observation  of  training  activities  and  focused 
on  trainees’  skill  acquisition  and  performance  and  their  drivers.  A  team  composed  of  personnel 
from  the  U.S.  Army  Research  Laboratory’s  (ARL’s)  Human  Research  and  Engineering 
Directorate  and  Camegie-Mellon  University  (CMU)  executed  the  evaluation. 

Personnel  responsible  for  planning  the  study  first  observed  2-week-long  training  cycles  delivered 
at  USAES  sites.  The  purpose  was  to  familiarize  themselves  with  the  training  in  its  current  form 
and  the  training  environment.  The  finding  supported  planning  and  preparation  for  3  weeks  of 
comprehensive,  systematic  field  observation.  This  included  collection  of  empirical  data  about 
the  temporal  organization  of  training  activities,  the  activities  of  training  personnel,  and  the 
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behavior  and  performance  of  trainees.  A  priority  was  collection  of  data  that  tracked  changes  in 
trainee  performance  as  a  function  of  training  activities  or  in  other  words,  learning.  These  data 
would  document  individual  trainee  performance  from  initial  mine  detection  efforts  to  the  level  of 
land  mine  detection  performance  achieved  at  the  time  of  certification.  The  training  events 
driving  trainee  learning  were  recorded  as  well,  and  evaluation  focused  on  examining  how  these 
events  related  to  Soldiers’  learning  trajectories  and  the  skill  levels  they  attained  at  the  end  of 
NET  cycles. 


1.5  Indications  of  Learning 

Two  common  metrics  used  to  investigate  and  assess  learning  are  success  rates  and  speed  of  task 
execution.  Success  rate  is  the  proportion  of  opportunities  for  which  an  individual  achieves  a 
designated  goal.  Learning  is  reflected  in  an  increasing  proportion  of  successful  trials  (to  an 
asymptotic  level)  as  task  practice  accumulates.  The  relation  between  performance  and  practice 
takes  a  curvilinear  form  typically,  as  illustrated  in  the  dark  green  function  in  figure  2,  relating 
success  rates  on  the  y-axis  scale  on  the  left  of  the  chart  to  number  of  practice  trials  on  the  x-axis. 


Figure  2.  Generic  learning  curves. 


Learning  can  also  be  measured  as  acceleration  of  task  execution  with  increasing  task  practice 
(Newell  &  Rosenbloom,  1981).  According  to  the  Power  Law  of  Practice  (Newell  &  Rosenbloom, 
1981),  individuals  typically  demonstrate  rapid  decreases  in  task  execution  time  initially  but  gain 
more  slowly  with  further  practice,  thus  producing  diminishing  performance  returns.  This  is 
illustrated  by  the  light  green  function  in  figure  2  relating  practice  to  task  execution  times  (shown 
on  the  right-hand  y-axis). 

It  is  important  to  note  that  these  descriptions  of  learning  trajectories  make  several  assumptions 
about  the  learning  environment.  First,  the  descriptions  assume  that  reliable,  accurate,  and  timely 
feedback  about  a  learner’s  performance  is  available  to  the  learner.  The  absence  of  this  critical 
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ingredient  can  impede  or  negate  learning.  These  characterizations  also  assume  that  learners 
engage  in  a  task  for  which  they  have  had  no  prior  experience  and  that  the  task  remains  the  same 
throughout  the  practice  period. 

This  evaluation  of  mine  detection  training  focused  on  two  measures  one  can  use  to  infer  a  mine 
detector  operator’s  skill  acquisition:  (1)  probability  of  detection,  and  (2)  task  completion  time. 
Additional  measures  of  interest  included  target  and  clutter  discrimination,  operators’  subjective 
ratings  of  confidence  in  locating  buried  mine  targets,  and  rate  of  advance  (ROA). 


2.  AN/PSS-14  (formerly  known  as  HSTAMIDS) 


2.1  Description 

The  AN/PSS-14  is  a  lightweight,  human-portable,  hand-held  mine  detector  that  can  detect 
metallic  and  low-metallic  mines  (see  figure  3).  The  major  components  of  the  AN/PSS-14  are 
battery  case  and  cable,  electronics  unit  (EU),  earpiece  and  cable,  control  grip,  wand  assembly, 
and  sensor  head.  The  AN/PSS-14  is  used  to  support  off-road  movement  of  units,  clear  lanes,  and 
help  identify  leading  edges  of  minefields.  Non-engineer  units  use  the  AN/PSS-14  to  confirm  or 
deny  the  presence  of  mines  in  support  areas  and  logistics  sites. 
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2.2  Theory  of  Detection 

The  AN/PSS-14  uses  two  different  sensors  for  the  detection  of  mines:  electromagnetic  induction 
(EMI)  or  metal  detection  (MD),  and  GPR.  Each  sensor  returns  a  unique  audio  signal  and  can  be 
heard  by  operators  individually  or  in  combination  with  the  other. 

MD  operates  on  the  principle  of  EMI,  employing  components  that  transmit  electromagnetic 
signals  and  receive  such  signals  to  detect  the  presence  of  conductive  material.  The  transmission 
component  sends  an  electromagnetic  signal  into  the  soil,  which  induces  a  secondary  magnetic 
field  in  conductive  objects  and  materials.  If  the  strength  of  the  secondary  magnetic  field 
(dependent  upon  the  composition  and  quantity  of  the  material)  is  above  a  set  threshold, 
algorithms  that  monitor  return  signals  produce  an  auditory  signal  indicating  the  presence  of 
conductive  material  (Department  of  the  Army,  2003).  The  metallic  components  of  mines  within 
the  range  of  the  EMI  signal  produce  MD  responses. 

GPR  can  locate  objects  buried  as  deep  as  6  inches  in  the  soil  by  transmitting  radar  waves  into  the 
ground  and  analyzing  the  reflected  waves.  As  the  search  head  passes  over  buried  objects  whose 
dielectric  constant  differs  from  that  of  the  surrounding  soil,  changes  in  the  radar  returns  occur. 
Algorithms  that  monitor  return  signals  and  register  these  changes  can  thus  reveal  locations  of 
buried  objects.  The  AN/PSS-14’s  generation  of  GPR  output  signals  depends  upon  more  than  the 
system’s  registration  of  changes  in  return  signals,  however.  The  MD  system  must  also  register 
the  presence  of  conductive  material  in  its  sensor  field,  although  the  strength  of  the  MD  signal 
does  not  need  to  reach  the  threshold  required  for  an  MD  output  signal. 

The  sensitivity  of  the  MD  and  GPR  systems  to  different  properties  related  to  mine  targets 
significantly  augments  the  system’s  detection  potential.  Co-location  of  both  signals  provides 
highly  constrained  information  for  inferring  the  presence  of  a  buried  land  mine.  Although  many 
buried  objects  (and  terrain  features  for  GPR)  can  produce  output  signals  on  one  or  the  other 
sensor  operating  independently,  far  fewer  objects  (other  than  mines)  will  produce  output  from 
both  sensors.  Moreover,  both  sensors  are  sensitive  to  variation  in  signal  dimensions;  the  MD 
subsystem  can  reveal  greater  or  lesser  quantities  of  conductive  material  and  the  GPR  subsystem 
can  reveal  the  spatial  dimensions  of  buried  objects.  These  capabilities  provide  information  that 
further  constrains  interpretation  of  output  signals  to  broad  categories  of  mines.  If  properly 
interpreted  by  a  well-trained  operator,  the  AN/PSS-14’s  responses  offer  innovative  and  powerful 
detection  capabilities. 

Mine  detection  with  this  system  depends  on  its  operator’s  ability  to  recognize  the  spatially 
distributed  patterns  of  auditory  output  from  his  equipment  produced  by  land  mines.  The 
equipment  and  the  information  it  provides  when  operated  correctly  offer  enormous  potential  for 
land  mine  detection.  However,  this  potential  can  only  be  realized  by  training  operators  to 
(1)  produce  reliable  and  valid  signals  by  applying  proper  operational  techniques,  and  (2)  interpret 
and  recognize  the  spatially  distributed  patterns  of  auditory  output  from  the  equipment  produced  by 
land  mines  to  infer  their  locations. 
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2.3  Component  Operational  Skills 

Operators  must  acquire  multiple  skills  to  effectively  operate  the  AN/PSS-14.  First,  before  the 
system  is  used  for  land  mine  detection,  procedures  for  preparing  the  system  must  be  performed 
properly.  Two  primary  operational  sub-skills  that  determine  a  Soldier’s  proficiency  with  the 
AN/PSS-14  beyond  proper  system  preparation  and  calibration  are  search  sweep  technique  and 
footprint  development.  Soldiers  must  also  learn  supporting  skills  such  as  safety  measures  and 
equipment  maintenance. 

2.3.1  System  Preparation  and  Sensor  Calibration 

An  important  functional  feature  of  the  AN/PSS-14  is  its  capability  to  adapt  to  varying  soil  and 
terrain  conditions,  which  vary  widely  throughout  the  world.  Exploiting  this  capability  involves 
training  operators  to  correctly  perform  preparatory  sensor  calibration  procedures  that  enable  the 
system’s  subsystems  to  adapt  to  the  characteristics  of  the  environment  in  which  the  system  will 
be  used  (Glumm,  Reinhart,  Sexton,  &  Waugh,  2001).  Before  using  the  detector,  the  operator 
must  first  prepare  and  calibrate  the  AN/PSS-14  to  filter  soil  scatter  and  eliminate  the  effects  of 
mineralized  soils  and  “train”  the  GPR. 

After  the  system  is  unpacked  and  assembled,  preparation  and  calibration  procedures  are 
performed  in  the  following  sequence:  (1)  attach  the  earpiece,  (2)  turn  the  detector  on,  (3)  allow 
the  detector  to  warm  for  5  minutes,  (4)  turn  the  detector  off,  (5)  wait  10  seconds,  (6)  turn  the 
detector  back  on,  (7)  perform  noise  cancel  to  establish  an  environmental  noise  baseline  for  the 
detector,  (8)  perform  ground  balance  to  eliminate  soil  mineralization  effects,  (9)  train  the  GPR  to 
filter  soil  scatter,  and  (10)  verify  the  calibration  process  with  a  test  piece  to  ensure  proper 
signaling  and  sensitivity.  Once  the  detector  is  successfully  calibrated,  the  operator  is  prepared  to 
search  for  mines.  Audio  signals  alert  the  operator  when  a  target  is  present.  The  operator  first 
finds  the  origin  of  the  metal  signature  and  then  uses  GPR  to  establish  whether  the  target  is  a  mine 
(returns  a  GPR  signal)  or  clutter  (no  GPR  signal). 

2.3.2  Core  Operational  Skill  Components 

2.3.2. 1  Search  Sweep 

Search  sweep  is  the  initial  technique  applied  in  detection  activities.  Its  purpose  is  to  identify 
locations  in  the  area  to  be  cleared  (USAES  training  lane  dimensions  are  1.5  m  by  15  m).  These 
locations  are  identified  by  an  output  signal  on  the  MD  channel,  the  GPR  channel,  or  both. 

Effective  search  sweep  involves  complete  spatial  coverage  of  the  designated  area  for  search  with 
the  sensor  head.  Complete  coverage  is  necessary  but  is  not  a  sufficient  condition  for  effective 
search  sweep.  Effective  search  sweep  sensor  head  movements  fall  within  the  kinematic  ranges 
needed  for  effective  sensor  operation.  Violation  of  these  thresholds  can  compromise  the 
sensitivity  of  the  system  sensors  and  result  in  missing  land  mines.  Even  though  the  sensor  head 
may  pass  over  a  mine,  if  it  is  too  high  or  moving  too  quickly,  no  return  signal  will  sound. 
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Proper  search  sweep  technique  is  the  first  of  the  primary  operational  skills  in  which  Soldiers  are 
trained.  The  multiple  requirements  for  effective  search  sweep  that  an  operator  must  learn  to 
produce  are  as  follow: 

•  Sensor  head  must  be  parallel  to  the  ground. 

•  Sensor  head  must  be  swept  as  closely  as  possible  to  the  ground  without  contact  and  never 
in  excess  of  the  2-inch  upper  limit. 

•  Sensor  head  trajectories  should  be  perpendicular  to  the  long  axis  of  training  lane  in  as 
straight  a  line  as  possible.  Trainees  are  explicitly  instructed  to  avoid  arc-shaped  paths. 

•  Sensor  head  velocity  with  the  lane  boundaries  should  be  with  the  1 .0-  to  3.6-ft/sec  range. 

•  Lane  traversals  continue  to  the  point  at  which  one-half  of  the  sensor  head  extends  outside 
the  lane  boundary. 

•  Forward  progress  should  proceed  with  trajectories  that  overlap  the  area  covered  on  the 
previous  traversal  by  one-third  of  the  sensor  head. 

Accurately  evaluating  novice  operator  movements  of  the  sensor  head  accurately  is  an  extraordi¬ 
narily  difficult  (if  not  impossible)  perceptual  task  for  trainers  and  observers.  Therefore,  to  provide 
accurate  performance  feedback  for  trainees  and  evaluative  information  for  trainers,  the  training  aid 
called  the  Sweep  Monitoring  System  (SMS)  (Herman,  McMahill,  &  Kantor,  2000)  was  made  an 
integral  component  of  AN/PSS-14  training.  This  computer-based  training  aid  collects  and 
graphically  displays  information  about  these  features  of  an  operator’s  search  sweep  for  use  by 
trainees  and  trainers,  providing  them  with  critical  performance  feedback  needed  for  effective  and 
efficient  training. 

2. 3. 2.2  Footprint  Development 

The  second  critical  operational  skill  that  AN/PSS-14  operators  must  acquire  in  training  is  the 
procedure  for  “footprint  development”  and  interpretation  of  the  procedural  results. 

A  land  mine  “footprint”  is  the  spatially  organized  pattern  of  auditory  signals  produced  by  a  hand¬ 
held  detector  when  it  senses  buried  land  mines.  After  the  origin  of  an  alerting  signal  obtained  in 
search  sweep  is  located,  this  location  serves  as  a  point  of  reference  for  acquiring  the  localized 
sensor  output  from  which  the  pattern  or  footprint  of  a  potential  mine  target  is  constructed  by  an 
operator.  The  operator  then  compares  the  resulting  pattern  to  the  patterns  of  mines  stored  in  the 
operator’s  knowledge  base  from  previous  encounters.  If  the  pattern  developed  is  recognized  as 
similar  to  that  of  a  mine  pattern,  that  judgment  leads  to  a  mine  declaration. 

Training  includes  instruction  in  the  sequence  of  procedures  that  are  used  to  build  footprints. 
Repetitive  application  of  these  procedures  on  training  mine  targets  provides  practice  in  their 
application  and  familiarizes  trainees  with  the  generic  patterns  of  AN/PSS-14  response  signals 
produced  by  mines.  This  latter  experience  builds  the  knowledge  that  is  the  basis  upon  which 
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operators  later  recognize  mine  patterns  and  make  “mine/no  mine”  decisions  in  training  and  in 
subsequent  live  operations. 

Footprint  development  procedure  involves  two  sequentially  organized  steps.  The  first  involves 
investigation  of  the  area  of  the  alerting  signal  with  the  MD  sensor  and  afterward,  the  GPR  sensor. 
The  MD  investigation  maps  an  area  around  the  origin  of  the  alerting  signal  whose  contours  are 
defined  by  the  onset  or  offset  of  output  from  the  MD  sensor.  This  area  defines  the  region  in  which 
the  operator  will  employ  the  GPR  sensor  to  verify  whether  the  suspected  target  is  a  mine. 

For  MD  investigation,  operators  are  instructed  to  move  the  detector  head  in  a  semi-circular 
fashion  around  a  suspected  target,  establishing  a  small  spiral  pattern  to  find  the  outer  boundaries 
of  the  target’s  signal  or  halo  (see  figure  4).  These  boundaries  are  defined  by  the  offset  or  onset 
of  an  MD  output,  depending  on  the  trajectory  of  the  sensor  head.  When  the  sensor  head  is  within 
the  MD  “halo,”  MD  output  persists;  movements  that  end  the  response  identify  a  halo  edge.  MD 
investigations  involve  sequentially  finding  MD  onset/offset  points  at  the  3-,  6-,  and  9-o’clock 
positions  of  the  MD  halo  by  continuously  tracking  its  edge  between  these  reference  points.  At 
each  halo  edge  point,  operators  mark  each  edge  position  by  placing  poker  chips.  These  visual 
markers  are  intended  to  support  operator  learning  so  that  they  can  later  visualize  the  MD  halo 
without  the  aid  of  the  physical  markers. 


When  the  MD  halo  is  completed,  its  geometry  directs  the  operator  where  to  proceed  with  GPR 
investigation.  In  initial  drills,  operators  are  instructed  to  place  a  chip  at  the  center  of  the  “clock 
face”  defined  by  the  3-,  6-,  and  9-o’clock  edges  of  the  MD  halo.  If  the  investigation  involves  a 
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mine,  the  strongest  MD  response  should  occur  at  this  point.  As  trainees  show  proficiency  in 
locating  the  center  of  MD  halos,  trainers  remove  the  requirement  of  chip  placements. 

This  MD  halo  center  point  serves  as  a  reference  for  the  first  part  of  the  GPR  investigation,  GPR 
“short  sweep”.  The  results  of  GPR  short  sweep  are  the  basis  for  identifying  the  source  of  the 
MD  signals  as  a  mine  or  clutter.  Short  sweep  consists  of  movements  of  the  sensor  head  that 
obey  the  technique  requirements  of  search  sweep  but  whose  length  is  limited  by  the  spatial 
extent  of  the  MD  halo  about  the  center  point.  The  outer  limit  for  sensor  head  movement  is  the 
MD  halo  boundary,  which  should  never  be  exceeded.  If  GPR  signals  reliably  occur  in  repeated 
short  sweeps,  the  decision  rule  operators  are  given  dictates  that  they  declare  a  mine.  The  absence 
of  GPR  signals  indicates  clutter  if  and  only  if  proper  GPR  short  sweep  technique  is  applied. 
Violations  of  proper  techniques  frequently  produce  GPR  signals,  leading  to  operators  making 
false  alarms,  that  is,  declaring  clutter  as  mines.  With  the  proper  short  sweep  technique,  many 
commonly  found  high  metal  objects,  such  as  a  nail  or  other  metal  fragment,  are  often  small 
enough  that  the  GPR  will  not  detect  them,  thus  allowing  an  operator  to  identify  such  items  as 
clutter. 

If  short  sweep  produces  GPR  output,  operators  are  instructed  to  extend  short  sweep  farther  from 
the  footprint  center  to  identify  the  outer  edges  of  the  mine  target  at  the  3-,  6-,  9-,  and  12-o’clock 
positions.  Locating  these  points,  defined  by  the  onset  and  offset  of  GPR  signals,  confirms  the 
likely  center  of  the  mine  target,  which  should  be  spatially  coincidental  with  the  MD  center  point 
for  most  mines  (and  all  mine  simulants  used  for  training)  with  no  adjacent  metallic  clutter.  Such 
confirmation  increases  the  likelihood  of  a  declaration  being  scored  as  a  correct  detection  or  a 
“hit.”  This  is  because  the  very  stringent  accuracy  standard  for  scoring  requires  that  poker  chips 
used  to  declare  mines  be  placed  over  or  immediately  adjacent  to  the  body  of  a  mine. 

Mine  footprints  vary  systematically  as  a  function  of  mine  category.  Figure  5  shows  typical 
footprints  for  targets  varying  in  size  and  purpose,  antitank  (AT)  versus  antipersonnel  (AP),  and 
varying  metallic  content,  metallic  (M)  versus  low  metallic  (LM).  Note  that  the  variability  in  the 
relations  of  the  MD  and  GPR  halos  varies  considerably  across  the  four  major  mine  categories 
(ATM,  APM,  ATLM,  and  APLM)  represented  in  the  set  used  for  training. 

In  figure  5,  the  black  circles  represent  outer  contours  of  mine  targets  (not  drawn  to  scale). 

Shaded  green  regions  indicate  MD  halos;  red  marks  indicate  where  trainees  are  instructed  to 
place  MD  halo  marker  chips.  Orange  marks  indicate  GPR  return  signals  and  are  expected  when 
the  sensor  head  follows  the  trajectories  shown  by  the  thin  black  lines.  Note  the  coincidence  of 
the  centers  of  the  MD  and  GPR  halos  at  the  center  of  the  mines.  Also  note  that  the  sensor  head 
trajectory  for  the  ATLM  (bottom  right)  extends  beyond  the  MD  halo — something  that  should 
not  be  done  operationally  but  is  illustrated  here  to  demonstrate  the  location  of  GPR  return 
signals. 
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Figure  5.  Generic  mine  footprints:  APM  (top  left),  ATM  (top  right),  APLM 
(bottom  left),  and  ATLM  (bottom  right). 


Adjacent  metallic  clutter  typically  distorts  the  symmetry  typical  of  the  MD  halos  of  mines  in  a 
clutter- free  environment.  The  degree  of  distortion  is  related  mainly  to  the  size  and  shape  of  the 
clutter  halo  and  its  proximity  to  a  mine’s  MD  halo.  The  shapes  of  MD  halos  for  clutter  are 
typically  more  variable  than  the  MD  halos  for  mines.  MD  clutter  also  generates  subtly  different 
audio  signals  (in  terms  of  frequencies  and  amplitudes)  than  those  produced  by  the  mine  simulants 
used  in  training.  A  significant  implication  of  clutter-related  distortion  to  MD  footprints  is  that  it 
increases  the  difficulty  of  correctly  inferring  the  location  of  the  MD  center  point  used  as  a 
reference  point  for  subsequent  GPR  investigation  and  declaration  chip  placement. 

2.4  Training  Site 

The  NET  occurred  at  two  different  sites  at  Fort  Leonard  Wood,  Missouri:  Training  Area  206  and 
the  “pole  bam”  (see  figure  6).  The  latter  was  a  relatively  new  site  with  a  one-sided  pole  bam  that 
provided  limited  cover  from  bad  weather.  Initial  observations  for  planning  data  collection  occurred 
at  both  sites.  Data  collection  was  conducted  only  at  the  pole  bam  site  because  training  that  occurred 
during  the  time  scheduled  for  evaluation  was  performed  only  at  that  site.  Preparatory  observations 
occurred  at  both  sites. 
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Figure  6.  Pole  barn  facilities;  west  facing  view. 


The  pole  bam  facility  included  five  sterile  (non-cluttered)  lanes  and  nine  off-route  or  tactical 
lanes.  All  lanes  were  1.5  m  wide  by  15  m  long  and  were  divided  into  10  equally  sized  cells.  The 
sterile  sand  lanes  were  intended  to  have  no  clutter,  but  several  items  of  clutter  could  be  found  in 
most  of  these  lanes.  The  tactical  lanes  were  in  the  wooded  area  surrounding  the  site.  All  had 
natural  soils  mostly  cleared  of  surface  vegetation.  These  lanes  contained  pre-existing  clutter  and 
clutter  placed  by  trainers,  although  clutter  locations  were  not  mapped  before  this  study.  All  lanes 
contained  buried  targets  that  included  mine  simulants  with  diameters  of  6,  9,  and  12  cm 
(representing  APLMs),  20,  25,  and  30  cm  (representing  ATLMs)  as  well  as  high  metal  M15s 
(ATMs)  and  Ml  6s  (APMs).  Although  all  lanes  had  nine  targets,  often  with  one  of  each  type  and 
two  mine  simulants  (SIM6s),  the  number  of  each  type  varied  for  some  of  the  lanes.  The 
implications  of  this  lane-to-lane  variability  in  target  content,  as  well  as  the  substantial  variability 
discovered  in  the  number  of  clutter  items  between  lanes,  are  discussed  later. 

2.4.1  Training  Aids  and  Functions 

Two  training  aids  are  integrated  in  the  training  program.  One  is  the  SMS  mentioned  in  the 
earlier  discussion  of  search  sweep  and  illustrated  in  figure  7.  The  second  are  scoring  grids, 
devices  used  first  for  emplacing  and  documenting  the  locations  of  mine  targets  in  lanes  when 
they  are  constructed  and  later  in  training  for  scoring  trainees’  mine  declarations. 

The  role  of  the  SMS  is  to  accurately  and  objectively  assess  the  qualities  of  search  sweep 
technique  and  provide  feedback  needed  for  Soldiers  to  leam  proper  technique.  The  motivation 
for  including  this  device  in  the  training  package  stems  from  the  extreme  difficulty  that  even 
highly  skilled  human  observers  experience  in  reliably  and  accurately  judging  whether  a  trainee’s 
sweep  is  within  acceptable  ranges  on  the  multiple  parameters  listed  earlier. 
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Figure  7.  Sweep  monitoring  system  (SMS). 


The  SMS’s  two  cameras  track  the  motion  of  the  detector  sensor  head  and  continually  document 
its  position  within  a  lane.  When  an  operator  moves  the  detector  too  quickly,  too  slowly,  too  high 
from  the  ground,  or  creates  a  gap  in  lane  coverage,  the  SMS  produces  a  voice  message  indicating 
the  violation.  The  SMS  records  the  head  position  data  continuously  over  time  as  well  as  video  of 
the  operator  sweeping.  The  data  are  displayed  on  a  color-coded  interface  that  maps  the  operator’s 
sweep  activities  (speed,  height,  gaps)  for  the  entire  drill.  This  graphical  interface  enables  trainers 
watching  the  SMS  monitor  during  the  drill  to  easily  detect  and  diagnose  technique  flaws  and 
enables  an  operator  to  review  his  search  sweep  and  identify  problem  areas. 

Scoring  grids  (see  figure  8)  are  used  to  determine  whether  mine  declarations  made  by  trainees  in 
training  and  certification  testing  can  be  classified  as  mine  detections.  The  devices  consist  of  a 
square  frame  made  of  plastic  tubing  with  strings  spaced  at  15-cm  intervals  and  extending  vertically 
and  horizontally  parallel  to  the  frame  components.  The  matrix  arrangement  of  strings,  each 
labeled  with  a  letter  (A  through  J)  on  the  horizontal  axis  or  a  number  (1  through  10)  on  the  vertical 
axis,  creates  a  reference  grid.  Lanes  are  constructed  with  reference  pegs  along  the  boundaries  so 
that  the  scoring  grid  can  be  reliably  placed  in  the  same  location  for  each  of  the  ten  1.5-m  cells  that 
compose  a  lane.  Mine  targets  are  emplaced  with  their  centers  at  specific  intersections  of  the  grid, 
denoted  by  a  unique  letter  and  number  combination  for  each  cell  (Cell  2,  B-9,  for  example).  Thus, 
by  measuring  the  distance  from  a  declaration  chip  to  the  grid  location  specifying  a  target  center, 
adjusting  for  the  target’s  radius,  and  applying  due  diligence  with  respect  to  grid  placement  and 
measurement,  we  can  accurately  and  easily  categorize  the  declarations  as  hits  or  misses. 
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Figure  8.  Scoring  grid. 

2.4.2  Training  Schedule/Sequence 

Training  documents  list  the  content  and  planned  delivery  sequence  for  course  components  as 
follow: 

•  Day  1  -  Classroom  (6  hours),  sweep  technique  demonstration  and  practical  exercise; 

•  Day  2  -  Startup  and  calibration  demonstration,  mine  footprint  demonstration  and  practical 
exercise,  1 .5  sterile  lanes,  two  2-minute  runs  on  the  SMS; 

•  Day  3  -  Tactical  lane  practical  exercise; 

•  Day  4  -  Tactical  lane  practical  exercise,  kneeling  and  prone  mine  detection  demonstration, 
written  test  review; 

•  Day  5  -  Written  and  performance  tests. 


3.  Method 


Evaluation  methods  focused  on  empirical  observation  of  trainees  engaged  in  AN/PSS-14  mine 
detector  operator  training.  The  collection  of  quantitative  data  documenting  trainee  performance 
was  prioritized  because  learning  occurring  over  the  training  periods  represents  arguably  the  best 
index  of  training  efficacy.  A  variety  of  qualitative  observations  was  collected,  some 
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systematically,  if  they  were  anticipated  based  on  events  and  observations  encountered  in  the 
earlier  training  familiarization  phase.  Other  unanticipated  but  significant  events  that  occurred 
during  the  formal  observation  period  were  noted  and  described  qualitatively  in  the  results  that 
follow. 

3.1  Participants 

Candidates  for  detailed  observation  were  drawn  from  Active  Army,  Army  Reserve,  and  Army 
National  Guard  units  from  Indiana,  Ohio,  Alabama,  Alaska,  and  Colorado,  who  were  scheduled 
for  three  separate  week-long  training  sessions  at  Fort  Leonard  Wood,  Missouri.  All  units  but  one 
deployed  by  January  2005;  the  remaining  unit  deployed  by  August  2005.  Six  operators  from 
each  session  were  randomly  selected  for  detailed  observation  for  the  duration  of  their  training. 

Participants’  ages  ranged  from  19  to  47  years  with  a  median  age  of  29.  All  Soldiers  selected  were 
male  except  for  one  female,  and  their  ranks  ranged  from  Private  (E-2)  to  Sergeant  First  Class  (E-7), 
with  a  median  rank  of  Sergeant  (E-5).  Of  the  18  Soldiers  who  were  observed,  7  had  no  previous 
mine  detection  experience  and  the  remaining  1 1  had  AN/PSS-12  training.  Of  the  1 1  Soldiers  with 
training  experience,  one  had  AN/PSS-1 1,  Minelab,  and  HSTAMIDS  training  experience,  and  one 
had  used  his  AN/PSS-12  experience  in  a  combat  environment. 

All  participants  provided  informed  consent  before  any  observation  as  prescribed  in  Army  Regula¬ 
tion  70-25  and  required  by  the  internal  review  boards  for  the  protection  of  human  subjects  of  the 
authors’  institutions,  ARL  and  CMU.  A  copy  of  the  consent  statement  is  shown  in  appendix  A. 

3.2  Apparatus  and  Materials 

Equipment  for  data  collection  consisted  of  digital  stop  watches  used  to  time  and  record  events  in 
trainees’  mine  detection  drills  and  certification  testing. 

Several  structured  forms  tailored  for  specific  purposes  and  training  exercises  were  designed  to 
facilitate  and  standardize  data  collection. 

3.2.1  Training  Experience  Survey 

This  is  a  participant  survey  to  collect  information  about  past  experience  with  mine  detection 
systems  to  include  the  type  of  mine  detector,  training,  and  if  the  individual  had  used  the  detector 
in  counter-mine  operations. 

3.2.2  AN/PSS-14  Ergonomics  Questionnaire 

This  is  a  questionnaire  for  participants  to  answer  demographic  questions  and  to  rate  the  level  of 
pain  or  discomfort  with  certain  parts  of  their  bodies  before  and  after  training. 
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3.2.3  Daily  Log 

This  is  a  form  used  to  record  time  duration  of  the  daily  activities  of  participants.  Activities  of 
participant  included  arriving  and  departing  site,  classroom  training,  demonstration  training,  mine 
detecting,  assisting  other  participants  or  instructor,  breaks,  and  lunch. 

3.2.4  Sweep  Drill  Worksheet 

This  is  a  form  used  to  record  the  time  trainees  spent  on  sweep  drills  and  support  “battle  buddies” 
engaged  in  these  drills. 

3.2.5  Iteration  Cover  Sheet 

This  is  a  form  used  to  record  the  participant’s  name,  detector  serial  number,  starting  and  stopping 
times,  data  collector’s  initials,  and  the  lane. 

3.2.6  Blind  Search  Observation  Data  Worksheet 

This  is  a  form  used  to  collect  the  timed  observations  of  participants  while  they  were  operating  the 
mine  detector  in  a  lane.  The  researcher  tracked  the  location  of  the  operator  in  the  lane  by  cell 
number.  Times  were  recorded  for  each  activity  that  the  operator  completed  such  as  search  sweep, 
MD  footprint  development,  GPR  verification,  GPR  footprint  development,  or  detector  recalibra¬ 
tion.  The  location  of  each  target  investigation  was  numbered  and  recorded  on  the  lane  map  and 
data  sheet.  After  each  investigation  resulting  in  a  mine  declaration,  data  collectors  asked  their 
respective  operators  for  a  subjective  confidence  rating  on  a  Likert  scale  (1  to  5  scale  range  in 
which  5  =  confident  that  the  target  is  a  mine,  1  =  confident  that  the  target  is  clutter)  and  recorded 
their  responses.  Events  such  as  quick  calibration  of  the  system,  noise  canceling,  presence  of  an 
instructor  or  trainer  assisting  the  operator,  and  any  other  events  or  instructor  comments  deemed 
significant  were  recorded. 

3.2.7  Mine  Lane  Maps 

These  are  maps  of  the  mine  and  clutter  locations  for  each  lane. 

Copies  of  all  forms  are  shown  in  appendix  A. 

3.3  Procedure 

Data  collection  began  in  the  morning  of  the  first  day  of  each  training  week.  Data  collectors 
initiated  recruitment  by  briefing  all  assembled  trainees  about  the  study’s  objectives,  the  voluntary 
nature  of  their  participation,  and  informed  consent  requirements.  Those  providing  informed 
consent  completed  the  training  experience  survey;  the  ergonomic  questionnaire  was  then 
administered.  Training  proceeded  with  the  classroom  phase  of  instruction. 
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Upon  completion  of  the  classroom  introduction,  six  randomly  selected  volunteers  were  each 
introduced  to  the  data  collector  assigned  to  observe  and  document  all  of  that  person’s  training 
activities.  Data  collectors  recorded  information  about  participants’  activities  in  a  daily  log. 

When  trainees  proceeded  to  initial  drills  on  sweep  techniques,  data  collectors  “shadowed”  their 
respective  trainees  as  unobtrusively  as  possible  to  observe  component  activities,  time  stamp  their 
starts  and  stops,  and  record  instructors’  comments.  At  the  end  of  the  drill  period,  another 
ergonomic  questionnaire  was  given  to  participants. 

On  the  second  day  of  training,  data  collectors  recorded  events  for  their  respective  trainees  in  daily 
logs,  beginning  with  start-up  and  calibration  demonstration  and  drills,  a  demonstration  of  footprint 
development  by  an  instructor,  and  trainees  engaging  in  the  footprint  development  exercises.  As 
initially  observed  in  training  familiarization,  footprint  drills  were  not  performed  as  focused 
exercises,  as  designed  in  the  initial  HSTAMIDS  training.  Instead,  the  footprint  development  drill 
was  combined  with  search  sweep  to  create  a  “blind  search”  exercise  in  the  “sterile”  lanes.  Obser¬ 
vations  were  therefore  recorded  on  the  Blind  Search  Observation  Data  Worksheets  for  subsequent 
practice  on  the  sterile  and  tactical  lanes.  To  better  anticipate  trainees’  activities  and  thereby  facili¬ 
tate  data  recording  in  blind  search  exercises,  lane  maps  were  constructed  showing  locations  of 
mine  targets  and  clutter  based  on  data  collectors’  observations. 

After  each  mine  declaration  made  by  each  trainee  undergoing  observation,  the  data  collector 
asked  for  a  subjective  confidence  rating  using  the  scale  described  previously  which  reflected  that 
operator’s  confidence  that  the  declaration  marked  the  location  of  a  mine  of  any  type.  Following  a 
trainee’s  completion  of  blind  search  on  a  given  lane,  data  collectors  observed  instructors’  scoring 
of  declarations,  noting  the  observations  associated  with  successful  detections  and  missed  targets. 

The  fifth  day  of  training  consisted  of  a  certification  test.  Data  collectors  again  used  the  same 
procedures  and  forms  for  events  as  those  used  in  previous  blind  search  exercises.  Trainees  were 
assigned  a  tactical  lane  that  they  had  not  previously  completed  in  training  and  were  required  to 
successfully  detect  all  the  mines  in  the  lane  in  order  to  pass.  If,  after  completing  the  test  lane,  the 
participant  did  not  detect  all  the  mines,  the  participant  was  assigned  another  tactical  lane  for 
retesting.  Only  perfect  detection  performance  in  the  first  or  second  test  met  certification 
requirements. 

Following  completion  of  field  observations,  data  collectors  entered  all  hand-coded  observations 
into  computer  files  for  subsequent  analyses.  Anticipating  later  analyses  by  target  type  (APLM, 
APM,  ATLM,  ATM)  or  clutter,  data  collectors  added  appropriate  categorical  labels. 

3.4  Anticipated  Results 

Several  results  were  anticipated.  First,  based  on  performance  in  AN-PSS-14  operational  tests 
(Santiago,  Locke,  &  Reidy,  2004),  one  of  the  authors’  prior  experience  with  HSTAMIDS 
operator  training,  and  anecdotal  reports  about  training  effectiveness,  the  tracking  of  trainees’ 
detection  rates  over  the  course  of  blind  search  practice  was  expected  to  reveal  learning  curves 
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that  peaked  in  the  97%  to  100%  range.  Because  the  training  was  expected  to  provide  trainees 
with  prior  knowledge  and  experience  at  footprint  development  before  blind  search  exercises,  it 
was  anticipated  that  this  prior  experience  would  produce  initial  trial  probability  of  detection  (PD) 
well  above  where  it  would  be  expected  to  be,  if  no  prior  training  had  been  given  in  those  sub¬ 
skills  (as  suggested  by  the  functions  in  figure  2).  It  was  expected  that  first  trial  PD  would  be  in 
the  70%  to  80%  range  and  would  increase  with  practice  to  the  97%  to  100%  range. 

With  feedback  and  review  of  any  missed  targets  with  trainers  present  to  provide  coaching, 
trainees’  increasing  familiarity  with  the  patterns  of  information  that  defined  mine  targets  led  to 
the  expectation  that  they  would  gradually  begin  to  distinguish  mine  targets  from  clutter. 

Trainee  ability  to  recognize  the  patterns  of  AN-PSS-14  output  that  defined  mine  targets  was 
expected  to  increase  with  practice,  and  with  the  feedback  and  coaching  they  were  expected  to 
receive,  gains  in  operators’  confidence  ratings  were  anticipated  as  a  function  of  practice. 

Regarding  the  tracking  of  trainee  MD  and  GPR  investigation  times  for  mine  targets,  a  trajectory 
conforming  to  the  power  law  of  practice  (illustrated  in  figure  2)  was  expected;  however,  specific 
ranges  were  not  expected  for  starting  times  or  ending  times  for  either  of  the  measures  since  these 
measures  had  never  before  been  collected  systematically. 


4.  Results 


4.1  Time  Distribution 

Data  from  the  daily  log  worksheet  were  aggregated  over  the  three  observation  periods  to  create  a 
weekly  time  distribution  to  examine  time  allocations  to  events  and  activities  occurring  within  the 
40-hour  block  of  instruction.  A  summary  of  time  allocation  to  various  events  is  shown  in  figure  9. 

Lane  time  is  defined  as  the  duration  an  operator  was  in  a  lane  engaged  in  blind  search  for  mine 
targets.  “Calibrate”  or  “sweep”  was  the  time  spent  in  calibrating  the  AN/PSS-14  or  performing 
sweep  drills  without  investigating  targets  (the  detector  was  turned  off  for  the  drills).  “Chip”  or 
“observe”  was  the  time  spent  in  placing  chips  for  another  operator’s  footprint  development  of 
targets  during  blind  search  or  watching  another  operator  use  the  detector.  “Class”  was  time  spent 
by  participants  in  the  classroom,  listening  to  lectures  or  performing  preventive  maintenance  checks 
and  services.  “Demo”  was  the  organized  time  that  all  operators  spent  watching  the  instructors 
demonstrate  specific  techniques.  “Lunch”  reflects  time  spent  on  the  daily  meal  break.  “Break”  or 
“waiting”  was  time  spent  warming  on  a  cold  day,  standing  around  and  talking  with  buddies, 
waiting  for  an  instructor  to  grade  a  completed  lane,  waiting  for  a  lane  to  open,  and  so  forth. 
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Figure  9.  Median  weekly  time  distribution. 

These  activities  took  35.6  hours  of  the  available  training  period  on  average.  Of  the  4.5-day 
training  week,  more  than  30%  of  the  time  was  consumed  by  lunch  periods  (4),  breaks,  and  waiting 
periods.  The  proportion  of  time  in  non-training  activities  exceeded  that  spent  practicing  mine 
detection  in  the  test  lanes',  note  that  the  lane  time  for  individual  trainees  ranged  from  a  minimum  of 

7.2  hours  for  one  operator  to  a  maximum  of  16.0  hours  for  another.  Chipping  and  observing  time, 
whose  instructional  value  is  suspect,  also  varied  widely,  from  1.4  hours  to  9.0  hours  for 
individuals.  Time  allotments  for  remaining  event  categories  showed  relatively  little  variability. 

4.2  Probability  of  Detection 

PD  measures  the  capability  of  an  operator  to  detect  the  locations  of  mines  and  in  this  case,  mine 
simulants.  PD  is  the  proportion  created  when  the  number  of  mine  declarations  scored  as  hits  is 
divided  by  total  number  of  mines  encountered. 

Figure  10  displays  overall  PD  for  all  trainees  in  the  observed  sample  as  a  function  of  practice. 

The  authors  created  the  dimension  of  practice  by  ordering  blind  search  trials  in  which  all  trainees 
engaged,  noting  that  in  most  cases,  the  first  two  trials  were  labeled  Footprint  Development  in  the 
program  of  instruction  (POI)  but  in  effect,  constituted  blind  search  trials  performed  in  the 
“sterile”  lanes.  Trial  7,  also  performed  on  a  tactical  lane  in  which  a  trainee  had  not  practiced, 
represents  each  trainee’s  first  attempt  at  qualifying  for  certification.  Results  of  re-test  trials 
given  to  those  who  failed  the  first  test  are  not  shown.  Error  bars  indicate  the  95%  confidence 
interval  of  the  proportion  (PD)  as  calculated  with  the  modified  Wald  method  (Agresti  &  Coull, 
1998).  Caution  should  accompany  the  interpretation  of  the  PD  and  confidence  intervals  because 
of  measurement  issues,  discussed  next,  which  threaten  their  validity. 
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Figure  10.  Probability  of  detection  versus  trial  number  for  all  mine  types. 


Two  features  of  the  function  shown  in  figure  10  are  noteworthy.  The  first  is  the  surprisingly 
high  initial  trial  PD.  The  second  is  the  decline  between  the  first  two  trials  and  the  third. 

Differences  in  PD  between  Trials  1  and  3  were  statistically  reliable  (z2,  N  =  233,  =  5.39,  p  <  .02), 
as  were  those  between  Trials  2  and  3  (z2,  N  =  282,  =  1 1.3 ,p<  .001). 

These  features  of  trainee  performance  were  examined  as  a  function  of  mine  category  because  the 
footprints  that  are  the  basis  for  accurate  mine  detection  with  the  AN/PSS-14  differ  considerably 
across  categories.  PD  as  a  function  of  practice  for  APLM,  ATLM,  APM,  and  ATM  targets  is  shown 
in  figure  11.  Trial  1  detection  rates  again  show  high  performance  levels,  with  the  slightly  lower 
performance  for  low  metal  targets  (more  difficult  to  find)  versus  high  metal  targets  (expected). 
Differences  between  second  and  third  trial  performance  vary  considerably  for  different  target 
categories.  PD  for  ATMs  show  the  form  of  classic  learning  curves,  curvilinear  monotonic  increas¬ 
ing  functions.  In  contrast,  sharp  declines  of  30%  and  20%  are  shown  for  APLMs  and  APMs, 
respectively,  between  Trials  2  and  3. 

The  declines  in  PD  between  Trials  2  and  3  motivated  examination  of  PD  as  a  function  of  sterile 
versus  tactical  lanes  (see  appendix  B  for  a  detailed  description  of  PD  as  a  function  of  trials  and 
lanes).  A  striking  contrast  was  the  difference  in  between-lane  ranges  in  PD  for  the  sterile  (.05) 
and  tactical  lanes  (.14).  An  analysis  of  the  differences  between  the  tactical  lane  with  the  highest 
overall  PD  (Lane  I,  PD  =  .93)  and  the  three  tactical  lanes  with  the  lowest  PDs  that  were  completed 
by  more  than  one  trainee,  produced  reliable  differences  between  this  lane  and  H  (PD  =  0.93, 

N  =  187,  z2  =  6.13,/?  <  .05),  M  (PD  =  0.82,  N  =  151,  z2  =  4.95, p  <  .05),  and  F  (z2  =  3.55 ,p<  .05). 
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Figure  1 1 .  Mean  probability  of  detection  versus  adjusted  trial  number  by  mine  type. 

Observations  of  training  and  testing  produced  two  significant  qualitative  findings  relevant  to 
interpretation  of  PD  and  the  related  confidence  intervals,  which  raised  concern  about  their  relia¬ 
bility.  First,  trainers  or  trainers  in  training  often  interrupted  trainees  while  they  were  performing 
footprint  development  exercises  and  blind  search  drills  in  the  tactical  lanes  in  order  to  model  the 
procedures  and  proper  technique.  The  interruptions  frequently  consisted  of  the  trainer  (or  a  trainer 
assistant)  seizing  control  of  the  AN/PSS-14  sensor  head,  performing  the  component  steps  of 
footprint  development,  and  placing  the  declaration  chips  on  targets.  All  declarations  were  scored 
and  recorded,  however,  as  if  they  were  produced  by  the  operator.  This  practice  represents  a  source 
of  measurement  error  likely  to  inflate  PD  for  initial  trials  artifactually. 

A  second  source  of  measurement  error  was  observed  related  to  scoring  conducted  by  trainers  and 
trainers  in  training  in  the  tactical  lanes.  Scorers  were  observed  to  score  trainees’  declarations  on 
most  occasions  by  visually  estimating  the  distance  between  a  placed  chip  and  a  target’s  reference 
point  on  the  scoring  grid  placed  on  a  lane  from  distances  of  4  to  8  feet  away,  rather  than  using  a 
tape  or  ruler  to  measure  distance.  An  instance  of  this  practice  was  captured  on  a  video,  and  it 
shows  a  trainer  making  “eyeball”  estimates  of  declaration-target  distances  at  speeds  (mean  4.4 
seconds,  min  =  2,  max  =  7  sec  per  “measurement”),  hardly  conducive  to  accurate  measurement. 

Scoring  accuracy  was  challenged  by  trainees  on  occasions.  In  three  observed  cases,  declarations 
judged  as  misses  by  trainers  performing  scoring  and  protested  by  the  operators  involved  were 
later  shown  to  be  hits  upon  a  later  careful  re-examination  by  observers.  Such  lax  scoring 
practices,  whose  incidence  beyond  the  observed  samples  is  unknown,  invite  measurement  error 
in  scoring.  Such  errors  raise  doubts  about  the  accuracy  of  PD  measures  and  produce  inaccurate 
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feedback  upon  which  trainee  learning  depends.  The  scoring  errors  documented  here  had 
negative  consequences  for  the  morale  of  the  trainees  involved. 

4.3  Confidence  Ratings 

Figure  12  summarizes  subjective  confidence  ratings  given  by  each  observed  trainee  for  all  mine 
declarations  as  a  function  of  practice,  distinguishing  ratings  given  for  mine  targets  and  those 
given  for  false  alarms  (clutter  mistaken  for  a  mine).  Error  bars  in  this  figure  represent  the  95% 
confidence  interval  for  each  mean.  Confidence  as  a  function  of  practice  for  each  of  the  four 
target  categories  is  shown  in  figure  13. 


Figure  12.  Mean  confidence  rating  versus  trial  number:  all  mines  versus  clutter. 


The  salient  feature  of  the  functions  shown  for  mines  in  figure  12  is  their  negligible  slopes.  These 
functions  and  related  summary  statistics  in  table  C- 1  show  that  the  mine  detection  practice 
provided  over  the  course  of  training  does  not  reliably  increase  an  operator’s  self-assessed 
confidence  in  his  or  her  capability  to  detect  targets — an  effect  contrary  to  the  gains  expected  from 
training.  Although  the  obvious  differences  in  confidence  ratings  for  targets  and  clutter  items 
suggest  that  operators  should  develop  some  sensitivity  to  the  signals  that  distinguish  targets  from 
clutter  early  in  training,  subsequent  practice  does  not  improve  this  sensitivity  from  its  baseline 
level.  Note  that  the  value  of  5  represents  that  an  operator  feels  “confident”  that  a  given  declaration 
marks  the  location  of  a  mine  target. 
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Figure  13.  Mean  confidence  rating  versus  adjusted  trial  number  by  mine  type. 


Figure  13  shows  differences  between  operator  confidence  for  declarations  related  to  metallic  and 
low  metallic  targets,  with  high  metal  targets  consistently  receiving  higher  confidence  ratings  over 
the  course  of  blind  search  trials.  Examination  of  the  summary  statistics  for  confidence  ratings  in 
table  C-l  confirms  no  effect  of  practice  for  ratings  for  APLM  or  ATLM  targets,  the  threats  that 
heavily  motivated  the  development  of  the  AN/PSS-14. 

4.4  Investigation  Times 

An  investigation  event  was  defined  as  an  operator’s  sequential  application  of  MD  techniques  and 
GPR  techniques  to  an  area  where  an  alerting  signal  occurred.  This  definition  ignores  the  order  of 
technique  application,  noting  that  trainees  are  instructed  to  first  perform  an  MD  investigation  on 
a  suspected  target  and  then  use  its  findings  to  guide  GPR  investigation.  Because  investigation 
times  include  sensor-specific  investigation  activities  as  well  as  acts  such  as  quick  calibration  or 
grip  adjustment,  investigation  time  in  such  instances  exceeds  the  sum  of  MD  and  GPR  investi¬ 
gation  times. 

Figure  14  illustrates  investigation  times,  MD  investigation  times,  and  GPR  investigation  times  as 
a  function  of  practice.  Medians  summarize  the  temporal  measures  attributable  to  the  consistently 
skewed  form  of  these  time  distributions.  Medians  were  calculated  after  removal  of  outliers, 
defined  as  times  that  were  greater  than  three  times  the  value  of  the  fourth  spread  beyond  the  75th 
percentile  value  (Hoaglin,  Mosteller,  &  Tukey,  1983).  Additionally,  the  lowest  percentile  and 
other  clearly  erroneous  times,  such  as  investigation  time  with  zero  duration,  were  trimmed  before 
median  calculations. 
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Figure  14.  Median  investigation  times  (total,  MD,  and  GPR)  versus  trial  number 
for  all  mine  types  and  clutter. 

Investigation  time  for  all  targets  (mines  and  clutter)  decreases  substantially  (57%)  with  practice, 
also  showing  a  curvilinear  trend  consistent  with  the  power  law  of  practice  (Newell  &  Rosenbloom, 
1981).  This  general  description,  as  well  as  the  magnitude  of  the  apparent  training  effect,  applies  to 
the  functions  for  investigation  times  for  targets  and  clutter  and  for  the  durations  of  MD  investiga¬ 
tions  for  targets  and  clutter.  Decreases  for  GPR  investigation  times  as  a  function  of  practice  tend 
to  be  much  more  modest,  about  half  the  percentage  decrease  seen  for  investigation  and  MD 
investigation  times. 

Results  from  fitting  the  power  curve  function  (y  =  Cxb)  to  the  temporal  investigation  time  measures 
are  shown  in  appendix  D  in  tables  D-4,  D-5,  and  D-6,  and  in  figures  D-4  through  D-18.  Despite 
high  variability  resulting  in  relatively  poor  fits,  three  noteworthy  consistencies  emerge.  First, 
comparative  examination  of  R  values  in  tables  D-4,  D-5,  and  D-6  shows  that  the  best  fits  are 
achieved  for  investigation  times  and  MD  times  related  to  mine  targets.  Second,  fits  of  GPR 
investigation  times  are  quite  poor  across  all  target  types  and  clutter.  Third,  fits  to  the  investigation 
times,  MD  times,  and  GPR  times  for  clutter  are  poor.  Examination  of  slope  estimate  (b)  magnitudes 
in  tables  D-l,  D-2,  and  D-3  shows  that  the  lowest  values  are  related  to  the  functions  for  GPR 
investigation  and  clutter  targets.  Inspection  of  figures  D-4  through  D-18  shows  consistently  greater 
dispersion  of  individual  data  points  from  the  predicted  values  at  each  practice  level  for  the  poorer 
fitting  functions,  with  one  noted  exception:  high  dispersion  is  characteristic  of  investigation,  MD 
investigation,  and  GPR  investigation  times  for  metallic  targets. 
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4.5  Rate  of  Advance 

One  measure  of  operator  performance  that  is  of  considerable  operational  importance  is  ROA. 
ROA  measures  the  speed  with  which  a  given  area  can  be  cleared  of  mines  and  made  safe  for 
maneuver.  Slow  mine  detection  and  clearance  compromises  operational  tempo  and  threatens  to 
desynchronize  various  tactical  elements  of  mission  and  disrupt  plans. 

The  areas  or  lanes  on  which  participants  performed  blind  search  exercises  were  1.5  m  wide  by 
15  m  long,  whose  perimeters  were  marked  and  whose  interiors  were  divided  into  ten  1.5-m  by 
1.5-m  cells.  Not  all  cells  were  swept  on  all  trials,  e.g.,  Trial  1  typically  involved  a  search  of  five 
contiguous  cells,  but  trainers  sometimes  truncated  the  first  and  latter  trials  on  an  arbitrary  basis. 
To  measure  advancement,  taking  into  account  the  different  areas  actually  covered  in  trials,  ROA 
values  are  presented  in  units  of  square  meters  covered  per  minute  (m2/min),  making  ROA  values 
independent  of  the  specific  area  covered. 

Figure  15  illustrates  the  median  rate  of  advance  for  all  participants  as  a  function  of  practice.  The 
medians  displayed  were  computed  after  the  removal  of  outliers,  defined  as  values  greater  than 
three  times  the  fourth  spread  from  the  75th  percentile  (Hoaglin,  Mosteller,  &  Tukey,  1983). 

Error  bars  indicate  the  95%  confidence  intervals  for  the  resulting  central  tendency  estimates. 
Despite  the  absence  of  any  speed  requirements,  a  trend  indicating  increased  coverage  speed  is 
seen  through  Trial  6.  A  regression  in  speed  was  observed  for  the  final  trial,  which  represents  a 
trainee’s  initial  attempt  to  qualify  for  operator  certification. 


Figure  15.  Median  rate  of  advance  versus  trial  number. 
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Summaries  of  ROA  as  a  function  of  variables  other  than  practice  are  presented  in  appendix  E. 
Included  are  descriptive  statistics  for  ROA  by  lane  (table  E-l)  and  by  participants  who  passed 
the  first  test  (Trial  7)  and  those  who  did  not  (table  E-2).  The  information  is  also  presented 
graphically  in  figures  E-l  and  E-2. 

A  noteworthy  finding  shown  in  figure  E-2  is  that  ROA  functions  diverge  after  Trial  3  for  trainees 
who  pass  the  initial  certification  trial  (Trial  7)  and  those  who  fail,  with  the  successful  operators 
showing  faster  ROAs.  The  unsuccessful  operators  also  show  a  greater  reduction  in  ROA 
between  Trials  6  and  7,  with  Trial  7  falling  slightly  below  the  ROA  that  this  group  demonstrated 
in  its  initial  trial. 

Noting  that  no  time  limit  was  imposed  on  testing  and  operators  were  told  to  take  all  the  time  that 
was  needed,  extreme  differences  among  operators  were  observed.  Some  completed  their  lanes  in 
as  few  as  35  minutes,  but  one  took  3  hours  45  minutes  to  complete  one  lane. 

4.6  Qualitative  Observations 

The  number  of  trainers  varied  substantially  from  week  to  week.  For  the  first  training  week,  the 
three  regular  NCO  trainers  were  supplemented  by  two  civilian  contractor  trainers  from  Cyterra, 
the  manufacturer  of  the  AN/PPS-14.  For  the  second  training  week,  there  were  no  civilian 
trainers.  In  the  third  training  week,  three  civilian  trainers  were  supplied  by  Battelle  Corporation. 
In  addition,  as  many  as  ten  trainer  assistants  (TAs)  in  one  week  and  as  few  as  one  in  another 
functioned  as  trainers  by  observing,  critiquing,  and  coaching  trainees. 

The  quantity  and  quality  of  training  that  trainees  received  varied  substantially  within  weeks  and 
between  weeks  for  several  reasons.  First,  TAs  were  Soldiers  who  had  trained  and  qualified  for 
operator  certification  only  the  previous  week  and  so  possessed  relatively  little  experience  with 
the  AN/PSS-14.  Their  training  skills  varied  considerably  as  well.  Numerous  instances  were 
observed  in  which  a  TA  seized  the  detector  head  from  an  operator  who  was  thought  to  exhibit 
poor  technique,  proceeded  to  investigate  a  target  from  a  kneeling  position  using  poor  technique, 
and  then  ended  his  demonstration  lesson  with  comments  such  “see  that?”  or  “get  it?”  Different 
TAs  also  gave  contradictory  advice  about  technique  to  trainees,  which  often  resulted  in  trainee 
confusion  and  frustration. 

Problems  with  the  operation  of  the  SMS,  which  occurred  frequently,  reduced  the  pool  of 
appropriately  qualified  trainers  available  to  observe  and  coach  trainees.  Only  the  regular  trainers 
had  sufficient  familiarity  with  the  SMS  to  diagnose  and  (sometimes)  fix  the  problems. 

Simultaneous  responsibility  given  to  the  resident  NCOs  for  supervising,  coaching,  and  evaluating 
the  TAs  further  reduced  their  availability  to  work  with  the  trainees. 

Supervision  of  trainees  fell  dramatically  once  training  exercises  moved  to  the  tactical  lanes  for 
several  reasons.  One  is  that  the  tactical  lanes  were  dispersed  in  the  wooded  area  around  the  pole 
bam,  requiring  a  trainer  to  be  close  to  an  operator  to  observe  his  technique.  Because  as  many  as 
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six  operators  might  be  engaged  on  different  lanes  simultaneously,  experienced  trainers, 
especially  when  their  numbers  were  low,  could  only  split  time  among  operators  by  walking  from 
lane  to  lane.  Because  trainers  scored  completed  lanes,  this  task  further  reduced  their  availability 
for  observing  trainees. 

Finally,  during  test  and  practice  trials  on  the  tactical  lanes,  the  experienced  trainers  showed  a 
marked  tendency  to  congregate  under  and  around  the  pole  bam,  venturing  out  occasionally  to 
observe  and  coach  (if  deemed  necessary)  trainees  for  short  periods.  This  tendency  to  congregate 
under  the  shelter  of  the  bam  was  exaggerated  during  conditions  of  harsh  and  inclement  weather, 
which  dominated  for  the  three  observed  weeks.  In  fairness,  longer  periods  of  observation  and 
often  excellent  coaching  were  devoted  to  trainees  regarded  as  needing  coaching  on  the  basis  of 
their  performance.  At  the  same  time,  during  substantial  portions  of  trainee  activity  on  the 
tactical  lanes,  observers  frequently  noted  deficient  technique  as  well  as  safety  violations  that 
went  unnoticed  by  trainers,  even  during  certification  testing. 

Regarding  SMS  usage,  reliability  problems  resulted  in  frequent  down  time,  limiting  the  amount  of 
time  that  it  could  be  used  to  provide  trainees  with  the  feedback  needed  for  achieving  proficiency. 
This  shifted  the  burden  of  accurately  assessing  trainee  technique  (a  difficult  task  for  even  the  most 
experienced  personnel)  and  providing  feedback  to  the  available  trainers.  When  the  number  of 
trainees  engaged  in  sweep  drills  exceeded  the  number  of  available  trainers,  they  were  denied 
consistent  feedback. 


5.  Discussion 


This  study  was  undertaken  with  the  assumption  that  the  AN-PSS-14  training  conducted  at  the 
US  AES  was  effective  and  that  efforts  focused  on  identifying  means  to  improve  its  efficiency. 

Our  observations  have  raised  doubt  about  the  validity  of  our  assumption  and  issues  about  the 
effectiveness  of  AN/PSS-14  training  as  it  is  being  delivered. 

The  remainder  of  this  section  discusses  the  bases  for  questioning  training  effectiveness  found  in 
the  results,  organized  by  sequence  of  results  reported  in  the  previous  section.  Following 
discussion  of  results,  recommendations  for  changes  are  made  whose  implementation,  we  believe, 
can  raise  the  quality  of  AN/PSS-14  operator  training  to  the  standards  of  the  US  AES  and  the 
capability  of  operators  to  the  level  demanded  by  the  current  mine  threat. 

5.1  Time  Distribution 

A  major  concern  related  to  the  Army’s  adoption  of  the  AN/PSS-14  is  the  costly  40-hour  block  of 
training  required  for  its  operators.  Individual  operators,  however,  do  not  receive  40  hours  of 
training.  Results  from  this  small  sample  of  training  sessions  show  that  the  average  operator 
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spends  but  a  fraction  (22%)  of  the  40  hours  on  the  most  important  component  of  training:  mine 
detection  drills  (blind  search).  A  larger  proportion  of  each  cycle’s  training  time  is  expended  on 
lunch,  waiting,  and  breaks.  Realistic  task  practice  accompanied  with  accurate  and  timely 
performance  feedback  and  coaching  is  essential  for  skill  acquisition.  Because  maximizing  task 
practice  (within  the  available  boundaries)  is  the  key  to  maximizing  learner  capabilities,  and 
maximizing  the  returns  of  practice  requires  a  prior  base  of  knowledge  and  basic  skills,  increased 
individual  detection  practice  time  following  essential  preparatory  instruction  and  drills  should 
improve  training. 

Surprisingly,  little  time  was  spent  on  footprint  development,  the  core  and  most  complex 
component  of  AN/PSS-14  operation.  The  extent  of  instruction  focused  explicitly  on  this  critical 
subskill  was  a  15-  to  20-minute  demonstration,  although  in  one  week,  roughly  half  of  that  time 
was  used  to  demonstrate  footprint  development  for  only  two  of  the  four  mine  types.  Moreover, 
footprint  development  exercises  were  eliminated.  Instead,  as  described  earlier,  footprint  develop¬ 
ment  practice  was  merged  with  search  sweep  into  blind  search  drills  with  sporadic  and  often 
inconsistent  instructor  guidance  while  the  operator  practiced.  Few  Soldiers  appeared  to  execute 
the  techniques  involved  correctly  without  direct  instruction  from  trainers.  In  the  case  of  the 
training  period  with  a  severely  shortened  instructional  demonstration,  observers  from  the  Night 
Vision  and  Electronic  Sensors  Directorate  of  the  U.S.  Army  Research,  Development,  and 
Engineering  Command  who  were  present  (an  NCO  and  a  civilian)  intervened  and  initiated 
individualized  footprint  development  tutoring  for  trainees  in  response  to  the  obvious  confusion 
they  exhibited  in  executing  this  task  in  the  context  of  blind  search  drills. 

The  time  allocated  for  sweep  and  system  calibration  drills  appeared  sufficient,  based  on  trainee 
performance,  although  the  amount  of  time  spent  on  the  SMS  during  sweep  drills  was  sharply 
limited  because  of  the  unreliable  operation  of  this  system.  Because  the  performance  feedback 
that  the  SMS  provides  (when  working  properly)  is  likely  far  more  accurate  than  that  a  trainer  can 
provide,  maximizing  the  proportion  of  SMS  time  with  the  current  limits  of  that  allocated  for 
sweep  drills  is  desirable.  Given  the  purchase  cost  of  this  equipment  and  the  sunken  costs  related 
to  its  deployment,  its  limited  use  provides  a  very  low  return  on  investment. 

The  7-hour  block  of  time  of  classroom  environment,  even  with  interleaved  breaks,  challenged 
Soldiers’  ability  to  maintain  concentration  and  to  acquire  and  retain  the  volume  of  detailed 
information  presented.  Much  of  this  information  had  to  be  represented  outside  the  classroom  to 
enable  all  Soldiers  to  perform  drills  correctly,  which  suggests  information  overload  in  the 
classroom. 

Although  breaks  in  training  are  needed  for  recovery  from  physical  fatigue  and  relief  from  intense 
concentration  that  AN/PSS-14  operation  demands,  a  disproportionate  amount  of  trainee  time  was 
spent  off-task  for  lunch,  hourly  breaks,  and  waiting.  Chipping  or  observation  time  provided 
limited  benefit  to  participants;  often,  the  time  was  spent  watching  surrounding  activities  or 
chatting  with  a  buddy. 
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5.2  Probability  of  Detection 

Detection  rates  are  the  core  measure  of  operator  skill  and  the  effects  of  training.  Interpretation  of 
detection  rates  in  the  results  section  in  order  to  evaluate  training  effectiveness  is  problematic  for 
several  reasons.  Two  reasons  described  in  the  results  are  the  tendency  for  trainers  (or  assistant 
trainers)  to  intervene  in  blind  search  drills  and  make  declarations  for  trainees,  and  the  outrageously 
imprecise  declaration  scoring  procedures  used.  These  raise  questions  about  the  accuracy  of  the  PD 
measures  recorded. 

In  addition,  PD,  especially  in  test  trials  (Trial  7),  is  likely  inflated  by  some  operator  strategy  of 
lowering  the  criterion  for  making  a  mine  declaration  and  increasing  the  number  of  declarations 
placed,  thus  increasing  the  likelihood  of  hits  along  with  the  false  alarm  rate.  Trainees  talked 
openly  about  using  this  strategy  to  “pass  the  test”  and  receive  certification.  In  one  observed 
instance,  a  participant  placed  multiple  chips  near  each  declaration  in  hopes  that  at  least  one  chip 
would  be  near  enough  to  count  as  a  hit.  It  worked;  63  chips  were  laid  to  mark  the  nine  mines  (and 
miscellaneous  clutter),  enabling  the  participant  to  pass  the  test.  Another  test  strategy  involved 
exploiting  speed/accuracy;  the  ROA  results  show  a  distinct  decrease  in  ROA  from  Trial  6  to 
Trial  7,  while  PD  for  APLMs  and  ATLMs  shows  an  upward  inflection  at  the  same  point.  The 
absence  of  performance  standards  for  false  alarm  rates  or  ROA  invites  a  test-wise  shift  in  operator 
strategy.  The  strategies  that  appear  to  have  been  implemented  confound  interpreting  changes  in 
PD  as  a  function  of  practice  as  a  reflection  of  trainee  learning. 

Still  another  issue  challenging  the  interpretation  of  PD  measures  as  indicators  of  training  effective¬ 
ness  involves  test  conditions.  Test  lanes  differed  not  only  in  the  number  of  clutter  items  but  also  in 
the  proximity  of  that  clutter  to  targets.  Observations  reveal  that  the  presence  of  clutter  adjacent  to 
low  metallic  targets  increases  the  difficulty  of  their  detection.  Thus,  the  tests  taken  by  some 
operators  were  more  difficult  than  the  tests  taken  by  others.  It  follows  that  an  operator  who  can 
successfully  detect  all  targets  in  a  lane  with  few  clutter  items  adjacent  to  the  LM  targets  would  have 
a  much  easier  time  meeting  the  certification  criterion  and  could  do  so  with  a  lesser  level  of  skill 
than  if  he  had  been  assigned  a  lane  for  testing  on  which  clutter  items  were  adjacent  to  LM  targets. 

Acknowledging  the  scoring  and  test  strategy  issues,  6  of  the  18  operators  missed  mines  on  the 
first  test  (Trial  7) — a  costly  attrition  rate.  In  addition,  PD  for  the  group  on  low  metallic  APs  and 
ATs  on  Trial  7  is  well  below  the  certification  criterion.  These  observations,  plus  the  suspiciously 
shallow  slope  of  PD  function  between  Trials  3  and  6,  raise  concerns  about  training  effectiveness. 

PD  is  the  primary  basis  for  operator  certification.  The  observed  measurement  and  scoring 
problems  raise  concerns  about  the  validity  of  the  certification  process. 

5.3  Confidence  Ratings 

The  unchanging  confidence  ratings  for  clutter  and  mines  through  the  week  indicates  that  partici¬ 
pants  perceived  little  change  in  their  capability  to  detect  targets  or  distinguish  target  from  clutter 
as  a  result  of  the  instruction  and  practice  they  received. 
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The  clear  separation  between  the  functions  for  mines  and  clutter  in  figure  12  indicates  that  trainees 
acquired  some  sensitivity  to  the  difference  between  the  signals  produced  by  mine  targets  and  those 
produced  by  clutter.  However,  clutter  confidence  hovering  around  the  value  of  3  suggests  that 
participants  were  “confident”  that  clutter  items  were  targets  but  less  so  than  for  actual  targets. 
Participants  frequently  stated  a  confidence  rating  of  3  accompanied  by  the  comment  “There’s 
something  there;  I  just  don’t  know  what  it  is.” 

One  of  the  purposes  in  transitioning  from  the  AN/PSS-12  to  the  AN/PSS-14  was  the  added 
ability  to  discriminate  mines  from  clutter  through  the  use  of  GPR  and  to  consequently  reduce  the 
number  of  targets  that  must  be  confirmed  as  mines  by  probing.  Instructions  to  trainees  stated 
that  GPR  signals  occurring  over  the  center  of  a  metallic  halo  confirmed  a  mine  target. 

Conversely,  if  no  GPR  output  occurred  over  a  metal  signature,  then  the  target  was  probably 
clutter.  However,  if  the  operators  were  unsure  about  the  identity  of  a  target,  they  were  instructed 
to  mark  it  as  a  mine. 

Beyond  this  decision  rule,  clearly  specified  characteristics  for  correct  GPR  investigation  techniques 
were  not  given  to  trainees.  In  fact,  different  NCO  trainer  demonstrations  of  this  technique  over 
targets  showed  wildly  different  search  head  velocities;  this  variability  was  reflected  in  trainee 
techniques,  often  with  the  consequence  that  the  swings  artificially  produced  GPR  signals  over 
clutter  items. 

Another  phenomenon  confused  trainees  about  GPR  operation  and  interpretation  of  GPR  signals. 
Even  over  mine  targets,  the  production  of  GPR  signals  varied  from  swing  to  swing  with  the 
variability  clearly  related  to  two  factors:  the  characteristics  of  the  sensor  head  movement  and  the 
metallic  content  of  targets.  Regarding  sensor  head  movement,  increases  in  swing  velocity  and 
length  unheeded  by  the  operator  could  often  produce  GPR  output  where  previously  he  had 
received  none.  For  metallic  targets,  longer  GPR  investigations  over  the  high  metal  targets  often 
produced  an  increasing  number  of  GPR  returns  over  a  larger  area  than  occurred  initially.  Execu¬ 
tion  of  a  quick  calibration  operation  followed  by  resumption  of  GPR  investigation  of  the  target 
returned  GPR  output  to  the  original  levels  and  area.  This  variability  in  GPR  response  appeared  to 
confuse  operators. 

Signs  suggesting  confusion  included  statements  by  trainees  such  as  “What’s  wrong  with  the 
system?”  or  “What’s  going  on?”  Less  explicit  signs  took  several  forms.  Observation  showed 
numerous  occasions  when  an  operator  received  no  GPR  signal  but  marked  the  target  as  a  mine 
nevertheless  because  “there  must  be  something  there.”  In  other  instances  (even  seen  on  Trial  7, 
the  certification  test),  operators  performed  investigations  with  MD  only,  without  any  GPR 
investigation,  thus  ignoring  the  functions  of  GPR  sensing  and  using  the  AN/PSS-14  essentially 
as  a  metal  detector. 

We  attribute  the  inferred  confusion,  consistent  with  the  differential  confidence  ratings  for  M  and 
LM  targets  in  figure  13,  primarily  to  three  factors:  (1)  lack  of  explicit  training  guidelines  for  GPR 
investigation  technique,  (2)  absence  of  explicit  footprint  development  drills,  and  (3)  absence  of 
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timely  performance  feedback  telling  an  operator  if  a  mine  declaration  was  a  target.  Changes  in 
training  addressing  these  factors  are  included  in  the  recommendations  discussed  in  section  6.3. 

5.4  Investigation  Time 

One  approach  to  identifying  and  evaluating  operator  skill  development  throughout  the  training 
week  was  to  examine  changes  in  performance  and  subjective  confidence  as  a  function  of 
accumulated  task  practice.  The  expected  form  for  time-based  performance  measures  is  the 
exponential  decrease  illustrated  earlier  in  figure  2,  i.e.,  large  reductions  in  task  execution  times  in 
early  practice  trials  followed  by  continuously  diminishing  reductions  as  task  practice  accumulates. 

Results  show  that  learning  curves  of  this  form  are  observed  for  investigation  times  and  MD 
investigation  times.  The  GPR  functions,  on  the  other  hand,  suggest  minimal  learning  on  this 
component  of  AN/PSS-14  operation.  We  therefore  attribute  the  gains  in  overall  investigation 
times  largely  to  increasing  proficiency  in  MD  investigation  with  practice. 

Why  might  gains  in  MD  investigation  skill  occur  but  not  in  GPR  investigation?  Part  of  the 
answer  was  discussed  earlier;  training  content  and  procedures  offer  little  support  for  learning  to 
use  GPR  effectively.  In  contrast,  the  robustness  of  the  MD  system  produces  output  more 
reliably.  MD  signals  are  far  more  consistent  in  yielding  the  results  described  in  instruction  and 
demonstrated  by  trainers,  thus  providing  implicit  feedback  needed  to  support  learning.  The 
performance  feedback  given  after  each  trial  is  likely  to  lead  trainees  who  rely  heavily  on  MD 
signals  to  make  declaration  that  such  a  strategy  can  yield  acceptable  performance  and  enable 
them  to  meet  the  certification  standard  (especially  in  the  absence  of  any  false  alarm  criterion). 

In  short,  training  content  and  practice  may  unintentionally  lead  trainees  to  operate  the  advanced, 
very  capable  AN/PSS-14  much  in  the  same  fashion  as  its  predecessor  the  AN/PSS-12  and  other 
EMI  detectors  in  the  Army’s  inventory. 

5.5  Rate  of  Advance 

Several  factors  appear  to  contribute  to  improvements  in  ROA  that  occur  with  practice.  The 
improvements  seen  in  MD  investigation  times  are  one  factor;  increased  efficiency  in  search 
sweep  is  likely  another. 

It  is  unclear,  however,  how  well  ROA  measures  reflect  learning.  On  one  hand,  the  divergence  of 
tactical  lane  ROAs  for  trainees  passing  their  first  attempt  at  certification  suggests  that  ROA  is 
predictive  of  skill.  Evidence  for  a  speed-accuracy  trade-off  in  Trial  7  suggests  otherwise,  noting 
that  the  trainee  with  the  longest  lane  time  (3  hr  45  min)  passed  the  first  test.  It  is  reasonable  to 
ask  whether  individuals  whose  lane  coverage  times  exceed  those  of  other  successful  operator 
really  understand  the  fundamentals  of  AN/PSS-14  operation.  A  negative  feature  of  exceptionally 
slow  ROAs  is  that  lengthy  trials  stifle  the  flow  of  training  and  leave  operators  who  are  scheduled 
for  the  occupied  lanes  waiting  idly  and  unproductively  for  extended  periods. 


32 


Another  problem  in  interpreting  ROA  as  a  measure  of  an  operator’s  skill  is  the  variability  among 
lanes  in  the  amount  of  clutter  they  contain.  The  mean  number  of  clutter  items  per  lane  is  25,  but 
the  count  ranges  from  1 1  (Lane  N)  to  42  (Lane  H).  Generally  speaking,  more  clutter  items 
demand  more  investigations,  which  result  in  longer  trial  durations  and  lower  ROAs.  This  is 
reflected  in  an  inverse  relation  between  number  of  clutter  items  in  lanes  and  median  ROAs.  In 
short,  the  practical  implication  is  that  any  valid  comparison  or  interpretation  of  individual  ROAs 
must  control  at  least  for  the  number  of  clutter  items  in  a  lane. 

5.6  Performance  Feedback 

A  number  of  results,  including  those  suggesting  successful  learning  (MD  investigation  times), 
questionable  learning  (PD),  and  negligible  learning  (GPR  investigation  times),  can  be  explained 
in  terms  of  the  performance  feedback  provided.  Performance  feedback  (i.e.,  information  about 
the  success  or  failure  to  achieve  the  goal  set  for  a  task)  is  critical  for  effective  and  efficient 
learning.  Task  practice  without  accurate  and  timely  feedback  yields  negligible  learning. 

In  blind  search  trials,  feedback  consisted  mainly  of  an  operator  receiving  information  about  the 
number  of  mines  missed  after  scoring  occurred.  Locations  of  missed  targets  were  marked  and  the 
operator  returned  to  each  and  conducted  another  investigation  in  the  area  of  the  marker,  supervised 
and  critiqued  by  a  trainer.  Unless  a  target  was  missed,  operators  had  little  opportunity  to  associate 
the  patterns  of  signals  produced  by  individual  investigations  with  targets.  This  policy  provided 
little  useful  feedback  to  trainees  to  support  development  of  mine  detection  skills.  This  would 
account  for  little  improvement  in  PD  over  the  course  of  practice  on  the  tactical  lanes  and  the  flat 
confidence  functions.  The  limited  feedback  provided  minimal  opportunity  for  learning  to  differ¬ 
entiate  the  patterns  of  signals  produced  by  mine  targets  and  those  produced  by  clutter.  Trainees 
were  uncertain  whether  declarations  should  be  marked  as  targets  or  as  clutter,  as  was  evidenced  by 
the  confidence  ratings  they  gave  to  clutter  items. 

The  paucity  of  feedback  also  appeared  to  cause  morale  problems  that  likely  impeded  the  learning 
of  some.  Expressions  and  signs  of  frustration  as  to  what  was  in  the  ground  were  abundant. 

Doubt  and  frustration  often  turned  into  an  attitude  of  indifference. 

In  contrast,  for  MD  investigations,  implicit  feedback  about  performance  was  available,  and 
trainees’  exhibited  learning  curves  were  of  the  sort  expected. 

The  negligible  improvement  in  GPR  investigation  times  can  be  interpreted  in  terms  of  the 
feedback  available.  The  inconsistency  with  which  GPR  signals  could  be  produced  and  trainees’ 
lack  of  knowledge  about  the  buried  items  producing  GPR  signals  denied  implicit  feedback.  The 
absence  of  clear,  objective  criteria  for  what  constituted  proper  GPR  short  sweep  technique  made 
it  impossible  for  trainers  or  operators  to  assess  the  quality  of  the  search  head  movements 
executed. 


33 


5.7  Certification  Standards 

US  AES  certification  of  trainees  is  based  primarily  on  PD,  the  accurate  measurement  of  which 
involves  several  standards.  First,  the  performance  standard  requires  that  an  operator  find  all  nine 
targets  in  his  first  or  second  test.  Measurement  of  performance  (i.e.,  PD)  is  based  on  a  scoring 
standard:  to  be  scored  as  hits,  a  declaration  marker  (a  poker  chip)  must  be  placed  so  that  it 
overlaps  or  is  adjacent  to  the  body  of  a  given  target.  Valid  scoring  of  declarations  depends  on 
meeting  a  measurement  accuracy  standard:  that  the  distance  from  the  edge  of  a  marking  chip  to 
the  nearest  edge  of  a  target’s  body  is  accurately  measured. 

Problems  observed  and  related  to  each  of  these  standards  call  the  validity  of  the  certification 
process  into  question.  The  practical  implications  of  this  situation  are  that  “certified”  operators 
can  vary  considerably  in  their  capability  to  operate  the  AN/PSS-14  effectively  and  safely  and 
that  some  may  actually  fall  below  the  desired  level  of  proficiency  that  certification  procedures 
were  instituted  to  ensure. 

Obtaining  accurate  target-chip  measurements  was  problematic  for  several  reasons.  First,  as 
described  earlier,  quick  visual  estimation  rather  than  careful  measurement  predominated.  Such 
procedures  surely  introduce  error,  especially  if  they  are  made  in  4  seconds  by  a  grader  standing 
8  feet  away  from  and  5  to  6  feet  above  the  points  to  be  measured.  Second,  one  of  these  reference 
points  is  the  intersection  of  two  strings  on  a  scoring  grid.  The  accuracy  of  any  measurement  using 
that  intersection  point  depends  on  accurate  placement  of  the  scoring  grid.  Careful  placement  of  the 
scoring  grid  was  inconsistent.  Third,  mines  tend  to  migrate  beneath  the  ground  surface  after  burial 
and  there  are  several  indications  that  some  targets  did  so.  Care  must  be  taken  to  locate  their  actual 
centers  and  record  their  position  on  grading  sheets;  these  new  positions  should  serve  as  reference 
points  for  measurement.  No  such  notations  were  found  on  the  grade  sheets.  Instead,  in  some 
cases,  graders  probed  the  ground  to  find  the  mine  and  estimate  its  center  point.  Two  draw-backs 
make  this  solution  untenable;  the  ground  can  only  be  probed  when  it  is  not  frozen,  which  it  was  for 
several  days  during  the  last  week  of  observations.  Successful  probing,  when  possible,  creates 
another  problem;  it  leaves  visible  cues  indicating  the  locations  of  targets  for  later  trials.  Some 
graders  obscured  the  probe  marks  by  scuffing  the  ground  surface,  thus  substituting  visible  scuff 
marks  for  signs  of  probing!  Such  marks  were  noticed  and  clearly  used  by  trainees,  with  the  likely 
consequence  that  it  made  the  targets  in  question  easier  to  detect. 

In  short,  very  stringent  accuracy  standards  have  been  adopted  for  scoring  and  laudably  so;  equally 
stringent  standards  and  quality  assurance  should  apply  to  chip-target  measurement  distances. 
Indeed,  accurate  feedback  needed  to  support  trainee  learning  depends  first  upon  accurate 
measurements  and  scoring. 

A  second  issue  arises  regarding  the  scoring  criteria  adopted.  A  far  more  liberal  scoring  criterion 
that  adds  a  6-inch  halo  to  the  perimeter  of  target  bodies  was  employed  by  US  AES  and  for 
operational  testing  of  the  AN/PSS-14.  The  current,  stringent  scoring  criterion  could  account  for 
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the  failure  to  observe  the  PD  achieved  in  operational  testing  and  reported  by  Santiago,  Locke, 
and  Reidy  (2004). 

Issues  were  also  discovered  regarding  the  criteria  used  for  certification.  Specifically,  the  absence 
of  false  alarm  rate  or  test  time  standards  puts  at  risk  the  certification  of  operators  who  may  not  be 
qualified  to  operate  the  AN/PSS-14  in  operational  environments.  Omission  of  standards  for  these 
measures  invites  strategies  for  operators  to  “game”  their  way  to  certification.  Trading  speed  for 
detection  accuracy,  as  observed  and  noted  earlier,  can  produce  certification  under  the  current 
standards.  However,  application  of  this  strategy  in  the  operational  environment  would  be  likely  to 
compromise  operational  tempo,  possibly  threatening  mission  success.  A  strategy  that  accepts  high 
false  alarm  rates  to  achieve  100%  PD  in  the  test  environment  would  likely  pose  a  similar  problem 
in  live  operations,  since  the  time  needed  to  confirm  declarations  via  probing  would  absorb 
prohibitive  amounts  of  time. 

Finally,  application  of  existing  certification  standards  to  trainees  tested  on  different  lanes  applies 
the  same  standard  to  tasks  of  differential  difficulty.  This  is  because  the  lanes  used  for  testing 
differed  in  the  number  of  clutter  items  they  contained  by  a  factor  of  more  than  3;  the  test  lane 
with  the  lowest  number  of  clutter  items  contained  1 1  (lane  N),  and  the  test  lane  with  the  highest 
number  had  36  (lane  K).  If,  as  it  appears,  clutter  placements  adjacent  to  low  metal  targets 
increase  the  difficulty  of  their  detection,  lanes  with  higher  clutter  counts  are  more  likely  to 
produce  this  condition  (assuming  random  placement  of  clutter),  making  such  test  environments 
more  difficult  than  lanes  with  less  clutter.  One  practical,  hypothetical  consequence  of  de  facto 
differential  standards  is  that  meeting  the  100%  PD  standard  on  tests  that  are  easy  may  certify 
operators  who  lack  the  skills  needed  to  accomplish  missions  in  an  operational  environment. 
Alternatively,  testing  on  excessively  difficult  lanes  may  results  in  operators  failing  to  receive 
certification,  who  have  the  skills  needed  to  successfully  execute  counter-mine  operations.  The 
latter  “error”  scenario  reduces  the  pool  of  operators  available  and  wastes  resources  invested  in 
training  the  individuals  who  are  erroneously  denied  certification. 

Apart  from  the  PD  criterion  for  certification,  safety  violations  committed  by  trainees  constitute 
grounds  for  denial  of  certification.  Such  violations  committed  by  trainees,  some  of  whom 
received  certification,  were  observed  in  training  and  testing,  although  many  were  not  noticed  by 
trainers. 

5.8  SMS  Equipment 

The  problems  experienced  with  the  SMS  resulted  in  the  use  of  this  training  at  a  level  lower  than 
planned.  The  SMS  offers  a  very  powerful  tool  for  providing  feedback  critical  to  trainees’  learning 
of  effective  search  sweep  technique  and  providing  trainers  and  trainees  with  information  to 
identify  and  diagnose  technique  problems. 

The  reliability  problems  observed  had  two  negative  consequences.  First,  the  problems  denied 
trainees  feedback,  which  even  experienced  trainers  are  challenged  to  provide.  Second,  time 
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devoted  to  system  setup,  diagnosis  and  solution  of  its  operational  problems  by  NCO  trainers 
reduced  the  number  of  experienced  trainers  available  to  observe  trainees  in  ongoing  sweep  drills 
and  to  provide  feedback  and  diagnose  problems. 

Considering  the  purchase  cost  of  the  SMS  and  the  sunken  costs  related  to  its  deployment,  its 
limited  usage  represents  a  poor  return  on  investment. 

5.9  Limitations 

Limitations  of  this  research  include  two  main  issues.  The  first  is  the  issue  of  generality.  Quanti¬ 
tative  data  were  collected  from  a  relatively  small  sample  of  1 8  operators  participating  in  three 
different  training  sessions,  of  the  approximately  150  Soldiers  trained  each  year.  Second,  the 
collection  of  data  outdoors  in  challenging  weather  was  arduous  and  demanding.  Although  all 
involved  sought  the  highest  levels  of  accuracy,  some  errors  of  commission  and  omission  in 
recording  and  transcription  must  be  expected.  To  counteract  threats  to  validity  that  such  errors 
might  introduce  and  to  mitigate  their  effect  by  statistical  means,  we  have  attached  confidence 
intervals  to  the  applicable  results.  When  measurement  error  was  beyond  our  control,  that  is,  in 
PD  measures,  we  have  explicitly  noted  the  problems  posed  for  interpreting  the  results  in  the  text. 

Regarding  the  issue  of  generality,  it  is  important  to  note  that  many  of  the  significant  qualitative 
observations  made  during  the  3-week  sampling  window  (e.g.,  PD  scoring  issues,  minimal 
focused  footprint  development  instruction,  SMS  problems  and  under-use)  were  seen  not  only  in 
the  2  weeks  of  preparatory  observation  at  the  US  AES  but  also  in  a  training  session  conducted  for 
USMC  trainees  at  Camp  Lejeune,  North  Carolina,  by  Cyterra  trainers  in  which  one  of  the  authors 
participated.  This  convergence  undermines  arguments  that  the  observations  reported  in  this 
study  cannot  generalize  beyond  their  narrow  sampling  context. 


6.  Recommendations 


6.1  Curriculum:  Content  and  Time  Allocations 

The  recommended  distribution  of  time  for  tasks  throughout  the  week  should  include  an  additional 
category  for  footprint  development  and  eliminate  time  for  chipping  and  observing.  Figure  16 
shows  the  proposed  time  allotment  for  tasks.  The  6  recommended  hours  for  footprint  instruction 
should  include  multiple  demonstrations,  practice  time  in  a  sterile  area,  experimentation  time  where 
multiple  targets  can  be  laid  in  proximity,  and  practical  time  when  instructors  can  observe  operators 
in  an  unmarked,  low-clutter  lane.  Actual  lane  time  should  be  increased  by  5.5  hours  to  14.2  hours. 
Class  time  should  be  reduced;  shorter,  spaced  classroom  sessions  interspersed  with  hand-on  exer¬ 
cises  will  help  operators  to  better  remember  the  information  needed  to  practice  proper  techniques 
and  develop  sweep  and  footprint  skills.  Chipping  and  observing  should  be  eliminated  in  favor  of 
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practical  exercises.  “Wasted”  time  for  lunch,  especially  the  usual  inflation  of  allotted  time  related 
to  travel  off  site,  can  be  eliminated  if  the  lunch  break  is  limited  to  30  minutes  and  trainees  are 
required  to  bring  lunches.  More  disciplined  administration  can  easily  reduce  time  wasted  on 
inflated  break  times  and  waiting  for  lanes  or  equipment  to  become  available.  Time  allocations  for 
demonstrations  and  calibrate  or  sweep  drills  should  stay  the  same. 


Figure  16.  Recommended  weekly  time  distribution. 


6.2  Performance  Feedback 

Operators  need  more  supervision.  Part  of  the  difficulty  in  increasing  supervision  and  the  delivery 
of  immediate  feedback  is  the  limited  availability  of  instructors,  but  that  does  not  diminish  the  need 
for  prompt  feedback  on  what  the  operator  is  doing,  right  or  wrong.  Effective  and  efficient  skill 
acquisition  demands  it. 

These  statements  equate  feedback  with  accurate  feedback.  The  scoring  problems  and  SMS 
problems  observed  need  to  be  fixed  to  provide  trainees  with  the  needed  feedback. 


6.3  Training  Content 

The  most  important  substantive  change  recommended  in  training  content  is  the  inclusion  of 
footprint  development  and  allocation  of  time  to  its  instruction  and  practice.  Absent  footprint 
development  drills,  trainees  can  rarely,  if  ever,  know  that  the  MD  and  GPR  signals  received  in 
an  investigation  are  those  produced  by  a  mine,  unless  they  have  missed  a  target.  Denial  of  infor¬ 
mation  impedes  a  trainee’s  learning  and  undermines  effective  and  efficient  acquisition  of  mine 
detection  skill.  The  suggested  time  distribution  invests  6  hours  of  the  40-hour  instruction  week 
to  footprint  development  instruction  and  practice. 

Training  content  needs  to  be  standardized.  One  of  the  largest  difficulties  encountered  across  the 
3  weeks  of  observed  training  was  the  variability  with  which  instructors  were  present.  Each  of 
the  instructors  had  different  ideas  about  how  much  information  to  present,  how  the  detector 


37 


worked,  which  technique  to  use,  and  what  content  should  be  emphasized.  As  an  example, 
several  instances  of  instruction  promoted  the  use  of  chipping  an  inner  halo  on  high  metal  targets 
to  facilitate  finding  the  center  of  the  target.  The  outer  halo  that  is  normally  chipped  is  located  at 
the  first  audible  sound  the  detector  produces  surrounding  the  target  (for  high  metal  mines,  this 
halo  can  be  30  inches  in  diameter).  The  inner  halo  is  located  where  the  detector  tone  distinctly 
jumps  from  a  low  pitch  to  a  high  pitch.  The  trouble  with  the  inner  halo  technique  is  that  if  the 
operator  in  any  way  misunderstands  or  forgets  that  the  area  between  the  outer  and  inner  halos  is 
not  assuredly  safe,  someone  may  step  on  a  “hidden”  mine,  a  low  metal  mine  whose  signature  is 
hidden  within  that  of  the  high  metal  target.  Techniques  must  be  standardized  to  prevent  these 
mistakes  and  the  hazards  they  create. 

As  another  example,  instructors  differ  in  what  constitutes  proper  sensor  head  velocity  for  GPR 
short  sweep  technique.  In  the  absence  of  clear  and  accurate  specification  of  an  effective  and 
acceptable  range,  trainers  and  trainees  have  only  their  intuition  to  rely  upon  to  assess  technique. 
Unfortunately,  such  intuitions  can  be  contradictory  and  simply  wrong. 

The  long  continuous  block  of  classroom  instruction  on  the  initial  day  of  training  should  be 
shortened,  with  its  contents  redistributed  and  interspersed  with  hands-on  activities.  Doing  so 
would  better  support  operators’  retention  of  the  theory  and  procedures  presented,  which  are 
necessary  to  practice  and  learn  effective  techniques.  In  addition,  efficiency  would  be  gained; 
we  found  that  much  of  the  content  presented  in  the  continuous  block  of  instruction  was  being 
repeated  before  hands-on  exercises  because  trainees  had  not  retained  it.  Following  an  overview 
of  training  and  an  overview  of  the  system  and  its  operation,  the  content  of  each  subsequent 
instructional  segment  should  be  limited  to  just  the  information  needed  to  support  trainees’  proper 
execution  and  practice  of  the  activities  covered. 

An  explicitly  “scaffolded”  approach  to  instruction  (also  expressed  as  the  “crawl,  walk,  run” 
instructional  strategy)  will  facilitate  skill  development,  and  we  recommend  its  adoption.  Basic, 
component  skills  should  be  taught  in  isolation.  Tests  should  follow  each  instructional  unit  to  assess 
trainees’  proficiency  in  these  sub-skills.  Testing  at  each  stage  or  level  of  training  would  help 
instructors  determine  which  operators  are  struggling  early  in  the  week  so  that  focused  diagnosis, 
remediation,  and  additional  practice  can  be  given.  Such  early  intervention  allows  struggling 
operators  to  clarify  task  requirements  while  the  instructor  is  present,  specifically  to  answer  ques¬ 
tions,  coach  proper  technique,  and  provide  feedback.  Most  participants  did  not  seek  instructors 
with  questions;  they  only  asked  if  instructors  were  conveniently  near  or  if  an  instructor  had  already 
engaged  them  in  conversation — events  that  were  too  infrequent.  Practicing  improper  or  weak 
techniques  led  to  the  acquisition  of  “bad  habits,”  which  are  difficult  and  costly  to  reverse  later. 

Trainers’  effectiveness  is  diluted  by  the  dual  responsibilities  assigned;  they  are  to  instruct,  coach, 
and  assess  new  operators,  and  when  TAs  are  present,  they  are  to  instruct,  evaluate,  and  test  TAs. 
Training  personnel  should  be  assigned  and  dedicated  to  one  instructional  task  or  the  other.  Imple¬ 
menting  this  recommendation  carries  nontrivial  immediate  costs:  it  would  require  additional 
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qualified  trainers.  We  believe  that  the  benefits  that  would  accrue  by  enhancing  the  training  of  new 
operators  and  new  trainers  could  offset  and  justify  the  cost,  especially  if  it  decreases  the  trainee 
failure  rate. 

There  are  two  elements  of  training  in  which  instructors  and  trainees  are  handicapped  by  a  lack  of 
information  about  AN/PSS-14  operation:  system  calibration  procedures  and  GPR  short  sweep. 

In  both  cases,  the  gaps  in  knowledge  result  in  (1)  trainees  lacking  clear  descriptions  of  the 
techniques  that  become  their  goals  to  acquire  and  practice,  and  (2)  trainees  and  trainers  lacking 
clear  standards  for  assessing  trainee  performance  of  these  significant  functions.  Trainers  can 
only  “eyeball”  an  operator’s  technique  and  guess  whether  it  is  good  or  bad,  lacking  clear  and 
specific  guidelines  for  distinguishing  between  techniques  that  prepare  the  sensor  systems  for 
effective  use  and  those  that  do  not.  The  validity  of  any  feedback  they  provide  is  therefore 
suspect,  if  any  feedback  is  given  to  a  trainee. 

Instructions  describe  system  calibration  procedures  as  critical  for  safe  and  effective  operation  of 
the  AN/PSS-14.  However,  no  objective  empirical  standards  are  available  for  reliably  and  validly 
assessing  whether  the  procedures  executed  have  effectively  prepared  the  system  for  use. 

Acquiring  the  needed  information  to  assess  system  calibration  and  documenting  the  procedures 
for  doing  so,  beyond  its  benefits  for  training,  could  resolve  a  chronically  controversial  issue  with 
regard  to  system  preparation  procedures.  Some  parties  have  advocated  checking  sensor  response 
by  burying  the  test  piece  that  comes  with  the  system.  Others  have  argued  for  performing  these 
checks  with  the  test  piece  on  the  ground  surface.  Procedures  for  objectively  describing  the  state 
of  the  system,  following  along  with  delineation  of  states  that  produce  acceptable  sensor 
responses  and  those  that  compromise  sensor  performance,  could  provide  measures  that  resolve 
the  controversy  experimentally. 

Trainers  have  been  observed  to  give  operators  contradictory  information  about  appropriate 
sensor  head  velocities  for  GPR  short  sweep,  as  related  earlier.  This  situation  arises  because  no 
explicit  specification  of  the  velocity  ranges  required  for  optimal  and  valid  GPR  sensor  response 
is  available  to  trainers.  These  operating  thresholds  for  GPR  short  sweep  should  be  established 
and  validated  empirically  (if  they  have  not  been  already)  and  disseminated  to  trainers.  We  note 
that  an  experimental  investigation  of  this  issue  should  involve  a  factorial  design  that  includes 
manipulation  of  the  variables  of  sensor  head  roll  and  head  height  because  our  observations 
suggest  that  increases  in  these  dimensions  are  correlated  with  increasing  velocities.  Training 
personnel  describe  this  phenomenon  as  “cupping,”  which  some  sources  speculate  produces 
artifactual  GPR  responses,  which  in  turn,  lead  to  false  alarms. 

A  final  issue  related  to  operation  of  AN/PSS-14  GPR  is  the  changing  response  observed  during 
lengthy  investigations  of  high  metal  targets.  An  accelerating  frequency  of  GPR  response  occurs 
over  an  area  growing  in  size  around  a  target’s  location.  Trainers  report  observing  this 
phenomenon  in  previous  training  sessions.  One  of  the  authors  has  experienced  it  while  operating 
the  system  in  training  at  Camp  Lejeune.  Indeed,  training  personnel  described  “unofficial” 
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techniques  for  mitigating  the  difficulties  it  created  for  accurate  declaration  placements.  These 
latter  observations  suggest  a  generality  that  extends  beyond  the  context  of  this  study. 

The  features  of  this  phenomenon  are  similar  to  those  that  are  observed  when  GPR  sensitivity  is 
increased  manually.  Regardless  of  the  underlying  mechanisms,  if  this  phenomenon  can  be 
shown  to  occur  with  any  regularity,  trainers  and  trainees  should  be  clearly  informed  of  the 
features  by  which  it  can  be  recognized,  its  consequences,  procedures  for  preventing  it,  and 
appropriate  responses  to  its  appearance.  Otherwise,  confusion  and  loss  of  confidence  in  the 
equipment  are  the  likely  responses  of  new  operators. 

A  final  issue  with  regard  to  training  content  is  whether  instruction  for  discriminating  mines  from 
clutter  should  be  included  in  the  POI.  This  is  a  controversial  policy  issue  whose  resolution  is  a 
matter  for  USAES  leadership.  How  it  is  settled  has  important  implications  for  the  content  and 
duration  of  AN/PSS-14  operator  training. 

Training  operators  to  distinguish  clutter  from  mines  is  feasible.  Experienced  operators  have 
demonstrated  in  test  settings  that  a  low  false  alarm  rate  can  be  achieved  while  extraordinarily 
high  detection  rates  are  maintained.  In  addition,  an  enormous  body  of  scientific  knowledge  has 
established  effective  procedures  for  training  discrimination  skills.  In  many  instances,  it  has  been 
applied  in  military  training,  notably  in  training  personnel  to  make  ftiend/foe  decisions.  Moreover, 
Biederman  and  Schiffrar  (1987)  have  demonstrated  that  techniques  used  to  analyze  expertise  in 
discriminating  between  complex  perceptual  categories  produce  knowledge  shown  as  highly  effec¬ 
tive  for  training  novices.  This  same  approach  has  been  applied  to  designing  the  currently  used 
AN/PSS-12  and  AN/PSS-14  training  programs  and  will  be  applied  to  a  proposed  study  of  the  best 
AN/PSS-14  operator. 

Although  we  believe  that  Soldiers  can  be  trained  to  develop  clutter  discrimination  skills  without 
sacrificing  high  detection  rates  (and  safety),  the  costs  of  doing  so  are  unknown.  If  the  recommen¬ 
dations  proposed  here  are  implemented,  however,  especially  those  that  can  increase  training 
efficiency,  these  costs  can  be  reduced. 

6.4  Certification  Standards 

Several  changes  should  be  implemented  to  achieve  the  effects  intended  by  instituting  an  operator 
certification  policy. 

First,  procedures  for  measuring  target-declaration  distances  must  be  specified,  monitored  for 
quality  assurance,  and  enforced  with  standards  as  stringent  as  the  accuracy  standards  demanded 
of  trainee  declarations.  Greater  use  of  tape  measures,  careful  placement  of  scoring  grids, 
accurate  recording  of  distances,  and  professionalism  in  the  application  of  all  costs  will  minimize 
measurement  error. 

Second,  regarding  scoring  standards,  we  urge  adoption  of  principled  distance  criteria  for 
computing  PD  that  would  resolve  the  discrepancy  between  two  apparently  arbitrary  standards 
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applied  by  US  AES  and  discussed  earlier.  One  reasonable  basis  for  developing  the  needed 
standard  involves  the  distances  between  declaration  and  mines  of  various  sizes  that  maximize  the 
safety,  effectiveness,  and  efficiency  of  probing.  Allowing  that  “to  err  is  human,”  an  extra 
“cushion”  for  unavoidable  errors  in  taking  measurements  should  be  incorporated  in  setting  the 
standard. 

Third,  related  to  the  testing  environment,  testing  should  take  place  on  lanes  equated  for 
difficulty.  Although  imposing  the  same  design  standard,  especially  with  regard  to  the  density 
and  placement  of  clutter,  to  all  training  is  not  necessary,  such  uniformity  should  be  required  of 
lanes  used  for  certification  testing. 

Fourth,  operators  can  adopt  strategies  to  “game”  testing  and  achieve  certification  by  exploiting 
the  absence  of  any  time  limit  or  false  alarms.  We  recommend  setting  and  applying  reasonable 
standards  for  these  performance  dimensions  to  ensure  that  certification  procedures  validly 
distinguish  between  trainees  who  have  the  capability  to  safely  and  efficiently  accomplish 
counter-mine  missions  and  those  who  lack  such  capability. 

Introducing  reasonable  time  limits  in  training  as  well  as  for  testing  holds  two  other  potential 
benefits.  First,  limiting  lane  time  will  facilitate  more  accurate  scheduling  and  lane  use  by 
reducing  the  unproductive  waiting  time  that  occurs  when  a  trainee  takes  extraordinary  amounts 
of  time  to  complete  a  lane.  Second,  lane  traversal  times  exceeding  1.5  to  1.75  hours  appear  to 
produce  levels  of  mental  and  physical  fatigue  that  result  in  trainees’  practicing  sloppy  tech¬ 
niques,  leading  to  skepticism  about  the  quantity  and  quality  of  learning  that  occurs  beyond  these 
durations.  We  propose  that  more  productive  practice  time  would  result  for  such  operators  if  such 
excessively  long  trials  are  truncated  and  the  operator  resumes  practice  on  another  lane  after  a 
mentally  and  physically  refreshing  break. 

6.5  Equipment 

Effective  demonstrations  of  AN/PSS-14  operations  require  that  trainees  see  the  detector’s 
movements  relative  to  targets  and  hear  its  output.  The  latter  often  proved  to  be  a  problem  when 
a  full  complement  of  trainees  (12)  gathered  around  the  trainer  giving  the  demonstration,  and 
especially  when  noise  from  other  sources  (e.g.,  wind,  machinery,  traffic,  aircraft)  was  present. 
The  use  of  attaching  equipment  (e.g.,  amplifier,  speaker)  capable  of  clearly  broadcasting  the 
detector’s  auditory  output  over  a  wider  area  and  other  interfering  sources  would  remedy  the 
problem  and  enhance  the  instruction. 

Quality  assurance  issues  are  chronically  reported  for  the  low  metallic  training  mines.  Effective 
and  efficient  training  requires  targets  that  function  reliably.  If  the  vendor  for  these  training  aids 
cannot  or  will  not  remedy  the  problems  experienced  with  these  very  expensive  targets,  we 
recommend  searching  for  a  supplier  who  will  provide  targets  of  the  quality  needed. 
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The  SMS  was  underused,  being  used  only  for  4  minutes  per  operator,  two  runs  of  2  minutes 
each.  In  order  to  receive  full  benefit  from  this  device,  it  should  be  used  several  minutes  every 
day  to  provide  accurate  and  objective  feedback  to  support  trainees’  learning  and  to  assess  the 
development  of  their  sweep  skills.  Its  potential  training  benefits  are  considerable,  but  they  can 
only  be  realized  if  the  system  is  used.  Chronic  reliability  problems  have  made  trainers  (USAES 
as  well  as  Cyterra)  justifiably  reluctant  to  invest  the  time  needed  for  setup,  only  to  find  that 
breakdowns  prevent  its  use  and  diagnosing  and  fixing  the  problems  divert  time  that  could  be 
spent  more  productively  observing  trainees.  If  the  vendor  for  this  system  cannot  or  will  not  fix 
these  problems,  we  recommend  that  USAES  look  elsewhere  for  other  equipment  that  can  serve 
the  function  of  the  SMS. 

De-milled  mines  produce  responses  from  the  AN/PSS-14,  which  differ  from  those  produced  by 
mine  simulants.  The  availability  of  samples  of  de-milled  mines  for  use  in  training  to  supplement 
the  use  of  simulants  can  help  familiarize  trainees  with  the  subtle  differences  between  mines  and 
training  simulants  and  thereby  promote  transfer  of  the  skills  developed  in  training  to  the 
detection  of  live  mines  in  actual  counter-mine  operations. 

6.6  Training  Facilities 

Many  features  of  the  new  pole  bam  site  at  Fort  Leonard  Wood  make  it  an  excellent  training 
facility.  The  following  recommendations  are  made  with  the  goal  of  making  an  even  more 
effective  environment  for  mine  detection  training.  While  some  are  specific  to  this  site,  the 
majority  are  more  generally  applicable. 

•  Results  of  several  measures  suggest  that  lanes  differ  considerably  in  difficulty.  For 
implementing  a  “crawl,  walk,  run”  instructional  approach,  we  advocate  lane  designs  that 
differ  systematically  in  their  clutter  content.  For  initial  blind  search  trials,  sterile  lanes  are 
desirable.  Trainees  who  demonstrate  proficiency  in  these  lanes  should  graduate  to  lanes 
with  clutter,  half  of  which  is  placed  adjacent  to  targets.  Still  another  set  of  lanes  should 
pose  a  greater  challenge  with  more  clutter  items  and  more  challenging  placements. 
Implementing  this  recommendation  will  likely  involve  construction  of  additional  lanes. 

•  To  promote  transfer  of  training  to  natural  environments,  we  recommend  selection  and  use 
of  clutter  items  likely  to  be  found  in  the  operational  environment  (e.g.,  brass,  cartridge 
chain  links,  barbed  wire)  in  the  orientations  in  which  they  are  likely  to  be  found  (as 
opposed  to  driving  10  penny  nails  straight  into  the  ground,  as  has  been  done  in  pole  bam 
lanes).  In  addition,  the  positions  of  all  emplaced  clutter  should  be  recorded  on  maps;  such 
maps  facilitate  evaluation  of  trainees’  skills  and  especially  identification  and  diagnosis  of 
problems  that  they  experience  in  detecting  mines  and  clutter. 

•  Sterile  lanes  need  to  be  sterile.  This  means  ensuring  that  lanes  designated  as  sterile  should 
be  thoroughly  cleared  when  constructed  and  maintained  in  this  state  by  regular  monitoring. 
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•  Several  lanes  need  improved  drainage.  For  example,  sterile  lanes  C,  D,  E  flood  after 
heavy  rains,  making  them  unusable  for  training.  In  several  other  lanes  standing  water 
collects,  precipitating  repeated  built-in  test  (BIT)  failures  in  the  older  model  AN/PSS-14s 
used  for  training.  Subsequent  use  of  the  system  after  BIT  failure  requires  the  time- 
consuming  processes  of  start-up  and  sensor  calibration.  Repeated  BIT  failures  multiply 
these  costs  and  frustrate  operators. 

•  High  densities  of  natural  clutter  at  the  staring  points  of  several  lanes  required  trainees  to 
expend  time  searching  for  a  clutter- free  area  to  ground  balance  the  MD  subsystem. 

Clearing  the  area  in  front  of  each  lane  would  eliminate  this  unproductive  time  spent  in 
doing  so  and  thus  increase  the  time  available  for  productive  training. 

6.7  Program  of  Instruction,  Efficiency,  Costs,  and  Skill  Retention 

The  recommendations  from  this  research  have  been  incorporated  into  a  new  POI.  Priority  in  its 
design  has  been  given  to  achieving  a  level  of  training  effectiveness  consistent  with  USAES 
standards  of  excellence.  The  recommended  changes  in  training  incorporated  in  the  proposed 
program  borrow  time  saved  by  the  elimination  of  discovered  inefficiencies  observed,  allocating 
the  savings  to  increased  practice  time  for  trainees. 

With  regard  to  training  efficiency  and  costs,  the  redesign  still  assumes  a  40-hour  training  period 
for  training  as  many  as  12  operators.  If  and  when  empirical  evaluation  demonstrates  that  the  POI 
achieves  the  training  standards  of  USAES,  efforts  to  maximize  its  efficiency  and  minimize  its 
cost  should  be  pursued.  The  recommendations  also  entail  added  costs;  adding  lanes  entails 
acquiring  more  targets.  Increasing  the  number  of  appropriate  qualified  trainers  imposes  a  non¬ 
trivial  personnel  cost. 

Are  such  costs  justifiable?  The  true  effects  of  NET  training  on  mission  capability  for  operators 
of  the  AN/PSS-14  (and  its  cost  effectiveness)  need  to  be  assessed  and  studied  in  a  context  that 
extends  beyond  the  terminal  performance  of  trainees  completing  training  successfully  and  the 
costs  involved  in  producing  it.  The  skill  retention  functions  shown  in  figure  17  suggest  that 
initial  investments  that  produce  the  highest  levels  of  skill  may  reduce  requirements  for  refresher 
training  and  may  be  offset  in  the  long  run  by  lower  resource  expenditure  requirements  to 
maintain  operator  and  mission  capability. 
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Figure  17.  Skill  retention  functions. 

6.8  Future  Research 

Understanding  GPR  response  to  kinematic  variations  in  the  movement  of  the  search  head  in  the 
context  of  GPR  short  sweep  will  benefit  training.  We  recommend  that  the  appropriate  experi¬ 
mental  studies  be  undertaken.  A  clear  and  specific  understanding  of  the  movement  parameters 
that  optimize  the  reliability  and  validity  of  GPR  output  will  provide  trainees  with  clear  goals  for 
training.  It  will  also  provide  trainees  and  trainers  with  information  needed  to  assess  the  quality 
of  trainees’  GPR  short  sweep  and  fix  any  deficiencies. 

Research  should  be  undertaken  to  establish  criteria  for  determining  when  the  MD  and  GPR 
subsystems  have  been  properly  calibrated  by  operators  and  when  they  have  not.  Results  should 
strengthen  efforts  to  establish  procedures  for  doing  so  in  the  context  of  training.  Once  the 
procedures  have  been  validated,  they  should  be  disseminated  immediately  to  trainers  and 
certified  operators. 

The  changes  in  the  training  recommended  here  and  incorporated  in  the  POI  need  to  be  tested  to 
ensure  that  they  deliver  the  expected  gains.  We  emphasize  that  rigorous  evaluation  would  need 
to  involve  evaluation  of  trainer  implementation  of  the  POI;  training  content  cannot  achieve  its 
goals  if  it  is  not  delivered  effectively. 

We  recommend  establishment  of  criteria  for  scoring  mine  declaration,  which  are  principled  and 
based  on  consideration  of  how  different  criteria  interact  with  subsequent  doctrinal  procedures  for 
mine  clearance.  A  level  of  accuracy,  that  is,  the  distance(s)  between  mine  targets  and  operators’ 
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declaration,  that  best  accomplishes  mission  requirements  should  be  the  basis.  Studies  of  probing 
effectiveness  and  efficiency  should  provide  critical  information. 

We  have  recommended  that  changes  in  operator  certification  criteria  be  considered.  Cost/benefit 
analyses  of  any  such  changes  ordered  and  implemented  should  be  undertaken  to  ensure  that  the 
expected  benefits  accrue  and  that  they  are  justified  by  the  investments  required. 

Following  any  changes  made  in  USAES  AN/PSS-14  training,  skill  retention  studies  should  be 
performed.  The  true  effects  of  NET  training  on  mission  capability  for  operators  of  the  AN/PSS- 
14  and  its  cost  effectiveness  need  to  be  assessed  and  studied  in  a  context  that  extends  beyond  the 
terminal  performance  of  trainees  completing  training  successfully  and  the  costs  involved  in 
producing  it.  Initial  investments  that  produce  the  highest  levels  of  skill  may  reduce  requirements 
for  refresher  training  and  may  be  offset  in  the  long  run  by  lower  resource  expenditure 
requirements  to  maintain  operator  and  mission  capability. 


7.  Conclusion 


The  objective  of  this  effort  was  to  examine  what  was  assumed  to  be  an  effective  training  program 
for  ways  to  increase  its  efficiency.  In  the  pursuit  of  this  goal,  deficiencies  in  AN/PSS-14  training 
were  discovered,  which  we  argue  undermine  its  effectiveness,  raise  doubt  about  the  validity  of 
operator  certification,  and  could  have  contributed  to  training  and  mission  capability  issues 
identified  by  two  in-country  evaluations  of  AN/PSS-14  operator  capability. 

In  addition  to  the  identification  of  training  problems,  principled  recommendations  for  changes  in 
the  AN/PSS-14  POI  that  represent  potential  solutions  are  offered  for  the  consideration  of  USAES 
leadership  and  other  stakeholders.  These  recommendations  describe  tactics  for  enhancing 
effectiveness  and  they  do  so  by  exploiting  time  gained  by  minimizing  the  inefficiencies  discovered 
in  the  current  training  program  implementation.  Priority  has  been  given  to  achieving  a  level  of 
training  effectiveness  consistent  with  USAES  standards  in  making  recommendations;  effort  to 
maximize  efficiency  and  minimize  costs  can  and  should  be  pursued  after  this  standard  is  achieved. 

The  AN/PSS-14  represents  an  important  technological  advance  that  holds  potential  to  substantially 
improve  the  counter-mine  capability  of  U.S.  ground  forces.  Multiple  studies  have  underscored  the 
critical  roles  played  by  operator  skill  and  operator  training  in  achieving  this  potential.  The  findings 
and  recommendations  reported  here  offer  a  path  for  advancing  toward  that  goal  that  is  cognizant  of 
needs  and  the  resource  limitations  under  which  USAES  works  in  meeting  these  needs. 
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Purl  B'lolw  rumplrtn)  by  the  Principal  InvcMigate 


Appendix  A.  Data  Collection  Forms 


Volunteer  Agreement  Affidavit 
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PSS- 1 4  NET  Trainee  Su  rvcv  SID  Code 


Training  Experience  Survey 
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Have  you  tivetf  used  the  PS§-14/H$TAMn)S  in  coyniennirLe  upecaEions 

leKcLuding  training)?  YES  NO 


A-N7PSS-14  Krgontmiics  Questionnaire 


AN/PSS-14  Ergonomics  Questionnaire 
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Turn  page  over  and  complete  the  buck  aide 


Time  Log  Worksheet 


AN/PSS-14  Training:  Time  Log 


Operator  ID:  _  Date: _ 

Location  examples:  inside,  SMS,  tactical  lane  206B  |  Pole  Barn 

(circle  one) 


Action  # 

Location 

Clock  start 
time 

Clock  finish 

time 

Activity/Comments 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 
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Sweep  Drill  Worksheet 


AN/PSS-14  Training:  Sweep  Technique 

Operator  ID:  _  Date: _ 

Battle  buddy:  _  Clock  start  time: _ 

Clock  finish  time: _ 

Sweep  location:  lines,  lane,  free 

Common  sweep  errors:  arc,  cup,  fast,  slow,  high,  gap,  level  206B  |  Pole  Barn 

(circle  one) 


Action 

# 

Location 

Start  time 

Finish 

time 

Inter¬ 

vention 

Problem 

fixed? 

Comments 

1 

Y/N 

Y/N 

2 

Y/N 

Y/N 

3 

Y/N 

Y/N 

4 

Y/N 

Y/N 

5 

Y/N 

Y/N 

6 

Y/N 

Y/N 

7 

Y/N 

Y/N 

8 

Y/N 

Y/N 

9 

Y/N 

Y/N 

10 

Y/N 

Y/N 

11 

Y/N 

Y/N 

12 

Y/N 

Y/N 

13 

Y/N 

Y/N 

14 

Y/N 

Y/N 

15 

Y/N 

Y/N 

16 

Y/N 

Y/N 

17 

Y/N 

Y/N 

18 

Y/N 

Y/N 

19 

Y/N 

Y/N 

20 

Y/N 

Y/N 

21 

Y/N 

Y/N 

22 

Y/N 

Y/N 

23 

Y/N 

Y/N 

24 

Y/N 

Y/N 

25 

Y/N 

Y/N 

26 

Y/N 

Y/N 

27 

Y/N 

Y/N 

28 

Y/N 

Y/N 

29 

Y/N 

Y/N 

30 

Y/N 

Y/N 
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Iteration  Cover  Sheet 


Cover  Sheet 

Data  Collector  Initials:  _ 

Operator  ID:  _ 

Battle  Buddy:  _ 

Date:  _ 

Start  Time:  _ 

End  Time:  _ 

Location:  206B  /  Pole  Barn 

AN/PSS-14  Serial  Number:  _ 

Task: 
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Quick  Cat 


Observation  Data  Entry 
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Mine  Lane  Map 
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Intentionally  left  blank 
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Table  B-l.  Descriptive  statistics:  probability  of  detection  by  target  type 


Appendix  B.  Probability  of  Detection 
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Figure  B-l .  Mean  probability  of  detection  versus  lane  for  all  trials. 


Figure  B-2.  Mean  probability  of  detection  versus  adjusted  trial  number  by 
participants’  test  result. 
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Appendix  C.  Confidence  Rating 


Figure  C-l.  Mean  confidence  rating  versus  lane  for  all  trials. 


Figure  C-2.  Mean  confidence  rating  versus  adjusted  trial  number  by  participants’ 
test  result. 
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Table  D-l .  Descriptive  statistics:  total  investigation  time  (seconds)  by  target  type, 


Appendix  D.  Investigation  Time 
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Adjusted  Trial  Number 


Figure  D-l .  Median  total  investigation  time  versus  adjusted  trial  number  by  mine  type. 


Figure  D-2.  Median  MD  time  per  investigation  versus  adjusted  trial  number  by  mine  type. 
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Figure  D-3.  Median  GPR  time  per  investigation  versus  adjusted  trial  number 
by  mine  type. 


Table  D-4.  Power  curve  (y  =  Cxb)  model  summary:  investigation  time  (y)  versus 
adjusted  trial  number  (x). 


Target  Type 

Model  Summary 

Parameter  Estimates 

R2 

F 

dfl 

df2 

Sig. 

C 

b 

APLM 

0.122 

44.088 

1 

316 

0.000 

149.887 

-0.394 

APM 

0.181 

17.052 

1 

77 

0.000 

230.628 

-0.489 

ATLM 

0.151 

60.751 

1 

342 

0.000 

178.576 

-0.467 

ATM 

0.115 

11.917 

1 

92 

0.001 

248.733 

-0.393 

Clutter 

0.044 

59.928 

1 

1,290 

0.000 

142.806 

-0.333 

Table  D-5.  Power  curve  (y  =  Cxb )  model  summary:  MD  time  per  investigation  (y) 
versus  adjusted  trial  number  (x). 


Target  Type 

Model  Summary 

Parameter  Estimates 

R2 

F 

dfl 

df2 

Sig. 

c 

b 

APLM 

0.116 

41.472 

1 

316 

0.000 

101.523 

-0.517 

APM 

0.165 

15.262 

1 

77 

0.000 

153.729 

-0.563 

ATLM 

0.129 

50.532 

1 

342 

0.000 

116.750 

-0.524 

ATM 

0.098 

9.997 

1 

92 

0.002 

170.595 

-0.458 

Clutter 

0.034 

45.976 

1 

1,290 

0.000 

83.189 

-0.377 
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Table  D-6.  Power  curve  (y  =  Cxb)  model  summary:  GPR  time  per  investigation  (y) 
versus  adjusted  trial  number  (x). 


Target  Type 

Model  Summary 

Parameter  Estimates 

R2 

F 

dfl 

df2 

Sig. 

C 

b 

APLM 

0.012 

3.695 

1 

316 

0.055 

26.804 

-0.173 

APM 

0.057 

4.619 

1 

77 

0.035 

39.587 

-0.345 

ATLM 

0.033 

11.739 

1 

342 

0.001 

32.771 

-0.319 

ATM 

0.028 

2.649 

1 

92 

0.107 

35.218 

-0.240 

Clutter 

0.011 

14.642 

1 

1,290 

0.000 

34.450 

-0.221 

Target  Type:  APLM 


Figure  D-4.  Total  investigation  time  versus  adjusted  trial  number  for  APLM  type  mines, 


Intentionally  left  blank 
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Appendix  E.  Rate  of  Advance 
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Figure  E-l.  Median  Rate  of  Advance  versus  Lane  for  all  Trials. 


Figure  E-2.  Median  Rate  of  Advance  versus  Adjusted  Trial  Number  by  Participants’ 
Test  Outcome. 
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